source context

Part 125

ctx:discord/blah/watt-activation/part-125

kind: custommode: permissive149 statements46 distinct subjects

No external document is attached to this context. (Many contexts are pure organisational labels.)

Facts in this context

Grouped by subject. Each subject links to its full article.

Xenonfun18 facts`ex:xenonfun`

asks	How Does That Change Things
attached	Screenshot 2026 03 08 11 50 26 Pm
attached	Screenshot 2026 03 08 11 54 04 Pm
expressesFrustration	No Checkpoints
expressesPreferenceFor	Tmux With Claude
isLettingRunTo	2k Seq Verify Inference
likes	Tmux Claude Setup
likesConvenienceOf	Tmux Switching
neverGot	Checkpoints
observesTooMuch	Vocab
postedAt	2026-03-09 03:33
postedAt	2026-03-09 03:54
postedAt	2026-03-09 03:57
postedAt	2026-03-09 03:55
postedAt	2026-03-09 03:50
sharesObservation	Vocab Issue
sharesStatus	Running to 2k
statesDesireFor	Seq 2048

Speaker14 facts`ex:speaker`

assertsMinimalPracticalChange	Seq Increase to 2048
assertsWithCertainty	Minimal Changes
evaluatesAsSafe	Bs 4 Seq 2048
instructsToUpdate	Seq 2048 Bs 4
needsToReCheck	Bs
offersAlternative	Go With Bs4
offersToMakeChangeAndPush	Script
offersToPushChanges	Script
offersToRun	Bs Sweep 4 8 Seq2048
presupposesPriorTesting	Bs 8 Seq 1024
proposesAction	Run Sweep
providesFullPicture	Memory Steps Context
seeksConfirmation	Confirm or Go With Bs4
summarizesRecommendation	Update to Seq2048 Bs4

Bs 4 Seq 20488 facts`ex:bs-4-seq-2048`

equalsTokensPerBatch	8192
hasIdenticalStepsAndEpochTimeAs	Bs 8 Seq 1024
hasSameMemoryFootprintAs	Bs 8 Seq 1024
hasSameStepCountAs	Bs 8 Seq 1024
hasSameTokSAs	Bs 4 Seq 1024
limitsBsMaxTo	4
losesThroughputAdvantageOf	Bs 8
maintainsSameTokensAs	Bs 8 Seq 1024

Training Command8 facts`ex:training-command`

logsEvery	100
savesEvery	2000
usesBs	4
usesCheckpointDir	Checkpoints Epoch Cl100k
usesLr	1e-04
usesSeq	2048
usesWarmup	500
validatesEvery	2000

Step 1007 facts`ex:step-100`

eta	175min
hasLoss	11.9294
hasLr	2.00e-05
hasPpl	151666.9
progress	0.6%
stepTime	635ms
tokPerS	12894

Step 2007 facts`ex:step-200`

eta	172min
hasLoss	11.4455
hasLr	4.00e-05
hasPpl	93477.9
progress	1.2%
stepTime	627ms
tokPerS	13067

Step 3007 facts`ex:step-300`

eta	172min
hasLoss	9.6110
hasLr	6.00e-05
hasPpl	14927.9
progress	1.8%
stepTime	630ms
tokPerS	13001

Step 4007 facts`ex:step-400`

eta	171min
hasLoss	8.7389
hasLr	8.00e-05
hasPpl	6241.2
progress	2.4%
stepTime	630ms
tokPerS	13007

Step 5007 facts`ex:step-500`

eta	169min
hasLoss	8.2263
hasLr	1.00e-04
hasPpl	3737.9
progress	3.0%
stepTime	628ms
tokPerS	13038

Lisamegawatts6 facts`ex:lisamegawatts`

advocatesWinningAt	Longer Context Even Harder
performedTestsOn	Memory Standard
postedAt	2026-03-09 03:59
postedAt	2026-03-09 04:00
predictsSuperiorityIn	Longer Context
reportsPersonalTests	Memory Standard Gpt Vs Spectral

Bs 16 Seq 10244 facts`ex:bs-16-seq-1024`

causedMemoryHitOf	85GB
hitMemoryUsage	85GB
knownToHit	85GB
presupposes85gbLimit	[object Object]

Model4 facts`ex:model`

learns	Longer Range Dependencies
presupposesHasGatedCumsum	[object Object]
presupposesHasPosEmbTo2048	[object Object]
trainsOn	2k Token Windows

Seq 20484 facts`ex:seq-2048`

changes	Context Length
enablesLearningOf	Longer Range Dependencies
isFreeUpgrade	Same Time Memory Better Context
providesBetterContext	[object Object]

Spectral4 facts`ex:spectral`

hasEssentialStability	Memory Standard
outperforms	Gpt
presupposesCompetitorToGpt	[object Object]
wasStable	Memory Standard

Training4 facts`ex:training`

hasSameTokensPerStep	Bs 8 Seq 1024
presupposesTotalTokensFixed	[object Object]
requiresEpochTime	~181 min
requiresSteps	~16,670

Bs 8 Seq 10243 facts`ex:bs-8-seq-1024`

equalsTokensPerBatch	8192
usedMemory	~40 GB
wasTested	~40 GB

Gated Cumsum3 facts`ex:gated-cumsum`

computesInLinearTime	Sequence Length T
handlesLongerSequences	Cheaply
hasTimeComplexity	O(T)

Bs 8 Seq 20482 facts`ex:bs-8-seq-2048`

causesDoublingOf	Activation Memory
hasSameLogitTensorSizeAs	Bs 16 Seq 1024

Epoch Time2 facts`ex:epoch-time`

approximately	181 min
equals181min	[object Object]

Spectral Plus Lohe V22 facts`ex:spectral-plus-lohe-v2`

benefitsFromLongerSeq	Meaningful
isMeaningfulFor	Longer Seq

Tmux2 facts`ex:tmux`

allowsSwitchingTo	Other Console Ctrl B N
presupposesSharedSessionWithClaude	[object Object]

Training Progress2 facts`ex:training-progress`

showsDecreasingLoss	[object Object]
showsDecreasingPpl	[object Object]

Bs 41 fact`ex:bs-4`

isSafeDefault

Seq 2048

Bs Increase1 fact`ex:bs-increase`

tradesOffWith

Seq Increase

Chat1 fact`ex:chat`

presupposesModelTrainingContext

[object Object]

Chat Log1 fact`ex:chat-log`

referencesAttachments

Screenshots

Checkpoints1 fact`ex:checkpoints`

presupposesExpectedButAbsent

[object Object]

Cl100k1 fact`ex:cl100k`

isCheckpointSuffix

[object Object]

Computation1 fact`ex:computation`

crunchedFor

31s

Eta1 fact`ex:eta`

decreasesOverTime

[object Object]

Gpt1 fact`ex:gpt`

caved

Memory Standard

Inference1 fact`ex:inference`

presupposesNeedsVerification

[object Object]

Logit Tensor Size1 fact`ex:logit-tensor-size`

scalesWith

Bs Times Seq

Longer Context1 fact`ex:longer-context`

hasTeleologicalValue

Learning Dependencies

Pos Emb1 fact`ex:pos-emb`

supportsUpTo

2048

Seq 10241 fact`ex:seq-1024`

allowedBsHeadroomTo

Seq 2048 Bs 41 fact`ex:seq-2048-bs-4`

isOptimalTradeoff

[object Object]

Sequence Doubling1 fact`ex:sequence-doubling`

doubles

Activation Memory

Spectral Lohe V21 fact`ex:spectral-lohe-v2`

isModelVariant

[object Object]

Step Count1 fact`ex:step-count`

approximately

16670

Tok Per S1 fact`ex:tok-per-s`

isStableAround

13000

Tok S1 fact`ex:tok-s`

roughlySameAs

Bs 4 Seq 1024

Total Steps1 fact`ex:total-steps`

equals

16670

Total Tokens in Flight1 fact`ex:total-tokens-in-flight`

equals

Bs Times Seq

Training Log1 fact`ex:training-log`

isFromRealRun

[object Object]

Training Run1 fact`ex:training-run`

totalSteps

16670

Part 125

Facts in this context

Xenonfun18 factsex:xenonfun

Speaker14 factsex:speaker

Bs 4 Seq 20488 factsex:bs-4-seq-2048

Training Command8 factsex:training-command

Step 1007 factsex:step-100

Step 2007 factsex:step-200

Step 3007 factsex:step-300

Step 4007 factsex:step-400

Step 5007 factsex:step-500

Lisamegawatts6 factsex:lisamegawatts

Bs 16 Seq 10244 factsex:bs-16-seq-1024

Model4 factsex:model

Seq 20484 factsex:seq-2048

Spectral4 factsex:spectral

Training4 factsex:training

Bs 8 Seq 10243 factsex:bs-8-seq-1024

Gated Cumsum3 factsex:gated-cumsum

Bs 8 Seq 20482 factsex:bs-8-seq-2048

Epoch Time2 factsex:epoch-time

Spectral Plus Lohe V22 factsex:spectral-plus-lohe-v2

Tmux2 factsex:tmux

Training Progress2 factsex:training-progress

Bs 41 factex:bs-4

Bs Increase1 factex:bs-increase

Chat1 factex:chat

Chat Log1 factex:chat-log

Checkpoints1 factex:checkpoints

Cl100k1 factex:cl100k

Computation1 factex:computation

Eta1 factex:eta

Gpt1 factex:gpt

Inference1 factex:inference

Logit Tensor Size1 factex:logit-tensor-size

Longer Context1 factex:longer-context

Pos Emb1 factex:pos-emb

Seq 10241 factex:seq-1024

Seq 2048 Bs 41 factex:seq-2048-bs-4

Sequence Doubling1 factex:sequence-doubling

Spectral Lohe V21 factex:spectral-lohe-v2

Step Count1 factex:step-count

Tok Per S1 factex:tok-per-s

Tok S1 factex:tok-s

Total Steps1 factex:total-steps

Total Tokens in Flight1 factex:total-tokens-in-flight

Training Log1 factex:training-log

Training Run1 factex:training-run

Xenonfun18 facts`ex:xenonfun`

Speaker14 facts`ex:speaker`

Bs 4 Seq 20488 facts`ex:bs-4-seq-2048`

Training Command8 facts`ex:training-command`

Step 1007 facts`ex:step-100`

Step 2007 facts`ex:step-200`

Step 3007 facts`ex:step-300`

Step 4007 facts`ex:step-400`

Step 5007 facts`ex:step-500`

Lisamegawatts6 facts`ex:lisamegawatts`

Bs 16 Seq 10244 facts`ex:bs-16-seq-1024`

Model4 facts`ex:model`

Seq 20484 facts`ex:seq-2048`

Spectral4 facts`ex:spectral`

Training4 facts`ex:training`

Bs 8 Seq 10243 facts`ex:bs-8-seq-1024`

Gated Cumsum3 facts`ex:gated-cumsum`

Bs 8 Seq 20482 facts`ex:bs-8-seq-2048`

Epoch Time2 facts`ex:epoch-time`

Spectral Plus Lohe V22 facts`ex:spectral-plus-lohe-v2`

Tmux2 facts`ex:tmux`

Training Progress2 facts`ex:training-progress`

Bs 41 fact`ex:bs-4`

Bs Increase1 fact`ex:bs-increase`

Chat1 fact`ex:chat`

Chat Log1 fact`ex:chat-log`

Checkpoints1 fact`ex:checkpoints`

Cl100k1 fact`ex:cl100k`

Computation1 fact`ex:computation`

Eta1 fact`ex:eta`

Gpt1 fact`ex:gpt`

Inference1 fact`ex:inference`

Logit Tensor Size1 fact`ex:logit-tensor-size`

Longer Context1 fact`ex:longer-context`

Pos Emb1 fact`ex:pos-emb`

Seq 10241 fact`ex:seq-1024`

Seq 2048 Bs 41 fact`ex:seq-2048-bs-4`

Sequence Doubling1 fact`ex:sequence-doubling`

Spectral Lohe V21 fact`ex:spectral-lohe-v2`

Step Count1 fact`ex:step-count`

Tok Per S1 fact`ex:tok-per-s`

Tok S1 fact`ex:tok-s`

Total Steps1 fact`ex:total-steps`

Total Tokens in Flight1 fact`ex:total-tokens-in-flight`

Training Log1 fact`ex:training-log`

Training Run1 fact`ex:training-run`