source context
Part 390
ctx:discord/blah/watt-activation/part-390No external document is attached to this context. (Many contexts are pure organisational labels.)
Facts in this context
Grouped by subject. Each subject links to its full article.
Xenonfun13 factsex:xenonfun
| agreesToAdd | yeah need to add that in |
| announcesLaunching | Sweep With All Configs |
| asserts | we got too synced |
| confirms | its already adamw only |
| explainsTechnicalDetails | Synchronization |
| performsActions | Sweep Relaunch |
| posesQuestion | with extra dim space could we not project other useful information? |
| reiterates | its too synced |
| sentMessageAt | 2026-03-19 02:32 |
| sentMessageAt | 2026-03-19 02:30 |
| sentMessageWithAttachment | Screenshot 2026 03 18 at 10 32 47 Pm Png |
| sentMessageWithAttachment | Screenshot 2026 03 18 at 10 36 19 Pm Png |
| sentMessageWithAttachment | Screenshot 2026 03 18 at 10 30 23 Pm Png |
Lisamegawatts10 factsex:lisamegawatts
| aimsToKeepOptimal | Lr |
| providesInsights | Decay Gates |
| sentMessage | ah you need to go adamw possible |
| sentMessage | no rotation |
| sentMessageAt | 2026-03-19 02:31 |
| sentMessageAt | 2026-03-19 02:36 |
| sentMessageAt | 2026-03-19 02:30 |
| suggestsAlternative | Kan |
| suggestsSwitchingTo | Adamw |
| suggestsUsing | that to derive lr |
Extra Groups7 factsex:extra-groups
| become | Computed Features of Byte |
| couldAdd | more sophisticated functions |
| couldEncode | Character Bigram Statistics |
| couldEncode | Byte Class |
| couldEncode | Hamming Weight |
| couldEncode | Nibble Level Features |
| currentlyGet | Xor Parity |
Model4 factsex:model
| hasHeads | true |
| learnsMemorySpans | Per Head |
| needsToLearn | short-memory (0.5) to long-memory (0.99) spread |
| wasStuckAt | uniform ~0.68 decay |
Sweep4 factsex:sweep
| expectedToShow | meaningfully better results |
| includesAllConfigs | true |
| relaunchedWith | Decay Fix |
| testsDecayFix | true |
Decay Gates3 factsex:decay-gates
| improvedPerformance | Clifford Gpt |
| madeBigDifferenceOn | Clifford Gpt |
| previouslyTested | Clifford Gpt |
Hamming Weight3 factsex:hamming-weight
| correlatesWith | Byte Value Distribution |
| isNumberOf | 1-bits (0-8) |
| usefulFeature | true |
R3 factsex:r
| approximates | 0.8 |
| isOrderParameter | true |
| measuresSynchronization | true |
Team3 factsex:team
| assumesSharedContext | Current Experiment |
| experiencedHandicap | Frozen Decay |
| plansOptimizations | Extra Group Features |
Adamw2 factsex:adamw
| currentlyUsed | true |
| isOptimizer | true |
Byte Class2 factsex:byte-class
| asOrthogonalDirection | true |
| is | ASCII category (letter/digit/punct/control) |
Current Run2 factsex:current-run
| hasNoRotation | true |
| referencesScreenshots | true |
Current Setup2 factsex:current-setup
| lacksRotation | true |
| usesAdamwOnly | true |
Decay Fix2 factsex:decay-fix
| enablesLearningOf | Per Head Memory Spans |
| improvesModelLearning | true |
Extra Dim Space2 factsex:extra-dim-space
| enablesProjectionOf | other useful information |
| existsForGGreaterThan8 | true |
Frozen Decay2 factsex:frozen-decay
| causedStuckState | Uniform Decay |
| wasSignificantHandicap | true |
Lr2 factsex:lr
| adaptsTo | R |
| derivedFrom | Sync Metric |
Nibble Level Features2 factsex:nibble-level-features
| includes | Upper Lower Nibble Values |
| like | Haar Wavelet Scales |
Order Parameter2 factsex:order-parameter
| servesAs | Learning Rate Signal |
| signalsLrAdjustment | true |
Oscillators2 factsex:oscillators
| haveSynchronization | R |
| performComputation | true |
Sweet Spot2 factsex:sweet-spot
| isEdgeOf | Synchronization |
| preferredForComputation | true |
Sweet Spot Groups2 factsex:sweet-spot-groups
| arePartiallyLocked | true |
| carryDistinctInformation | true |
Sweet Spot R2 factsex:sweet-spot-r
| rangesFrom | 0.3 |
| rangesTo | 0.6 |
Uniform Decay2 factsex:uniform-decay
| appliesAcrossAll | Heads |
| valueApproximately | 0.68 |
Byte1 factex:byte
| hasComputedFeatures | Extra Groups |
Character Bigram Statistics1 factex:character-bigram-statistics
| via | Phase Rotation |
Chat1 factex:chat
| involvesTechnicalDiscussion | ML Training |
Clifford Gpt1 factex:clifford-gpt
| priorExperiment | true |
Current Experiment1 factex:current-experiment
| isTooSynced | true |
Extra Group Ideas1 factex:extra-group-ideas
| isOptimizationForAfter | Scaling Results |
G Greater Than 81 factex:g-greater-than-8
| has | Groups Beyond 8 Raw Bits |
Groups1 factex:groups
| beyondRawBits | Computed Features |
Heads1 factex:heads
| haveDecayParams | true |
High Lr1 factex:high-lr
| enablesExploreMore | true |
Kan1 factex:kan
| alternativeArchitecture | true |
Long Memory1 factex:long-memory
| valueIs | 0.99 |
Lr Formula1 factex:lr-formula
| definedAs | lr = base_lr × (1 - r) + lr_min |
Memory Spans1 factex:memory-spans
| varyPerHead | true |
More Functions1 factex:more-functions
| asMoreGroupsAvailable | true |
Mostly Synchronized Oscillators1 factex:mostly-synchronized-oscillators
| perform | less differentiated computation |
Optimal Lr1 factex:optimal-lr
| maintainsPerformance | true |
Parity1 factex:parity
| rightIdea | true |
R 0 81 factex:r-0-8
| indicates | Oscillators Mostly Synchronized |
Raw Bits1 factex:raw-bits
| numberIs8 | 8 |
Scaling Results1 factex:scaling-results
| pending | true |
Short Memory1 factex:short-memory
| valueIs | 0.5 |
Synchronization1 factex:synchronization
| reducesDifferentiation | true |
That Run1 factex:that-run
| isTooSynced | true |
Xor Parity1 factex:xor-parity
| isComputedFunctionOf | Multiple Bits |