Lohe
From Dontopedia, the open, paraconsistent wiki. (Last updated 2026-06-06.)
Lohe has 142 facts recorded in Dontopedia across 28 references, with 8 live disagreements.
142 facts·118 predicates·28 sources·8 in dispute
Mostly:rdf:type(7), uses(6), is backprop free(2)
Maturity scale
raw canonical shape-checked rule-derived certifiedInbound mentions (40)
Other subjects in dontopedia point AT this entity as a value. These are inverse relationships — e.g. "X motherOf this subject" — and answer questions the forward facts can't. Grouped by predicate.
aboutAbout(1)
- Rotadamw Data Informative for Lohe
ex:rotadamw-data-informative-for-lohe
announcedDiggingIntoDetailedDataAnnounced Digging Into Detailed Data(1)
- Xenonfun
ex:xenonfun
announcedFoundBothProblemsAnnounced Found Both Problems(1)
- Xenonfun
ex:xenonfun
areCompletelyFrozenAre Completely Frozen(1)
- Oscillator Params
ex:oscillator-params
causesNoiseIssueForCauses Noise Issue for(1)
- Random Init
ex:random-init
changesBy3e-6PerStepChanges By3e 6 Per Step(1)
- Coupling
ex:coupling
collaboratesOnCollaborates on(1)
- Team
ex:team
decreasesMonotonicallyDecreases Monotonically(1)
- Val Ppl
ex:val-ppl
describesDescribes(1)
- Val Loss Trajectory
ex:val-loss-trajectory
describesComparisonDescribes Comparison(1)
- Finding 1
ex:finding-1
dominatedByBlockUpdatesDominated by Block Updates(1)
- Performance Metrics
ex:performance-metrics
expectedToImproveExpected to Improve(1)
- Pretrained
ex:pretrained
implyNonlinearExponentialBehaviorImply Nonlinear Exponential Behavior(1)
- Mathematics
ex:mathematics
includesOptimizerIncludes Optimizer(1)
- Phase 3
ex:phase-3
includesSeparateLrOscIncludes Separate Lr Osc(1)
- Fix Proposal
ex:fix-proposal
interferesWithKuramotoSyncInterferes With Kuramoto Sync(1)
- Gradient Normalization
ex:gradient-normalization
involvesOptimizerInvolves Optimizer(1)
- Smoketest Result
ex:smoketest-result
isActivatedByIs Activated by(1)
- Physics Gate
ex:physics-gate
isGradientNormalizationKillingMagnitudeIs Gradient Normalization Killing Magnitude(1)
- Problem 1
ex:problem-1
isLowerFidelityUpdateIs Lower Fidelity Update(1)
- Linearized Gradient
ex:linearized-gradient
isLr1e-4OverDampeningOscillatorParamsIs Lr1e 4 Over Dampening Oscillator Params(1)
- Problem 2
ex:problem-2
isPotentialImprovementOverIs Potential Improvement Over(1)
- H D Version
ex:h-d-version
measuredByValPplMeasured by Val Ppl(1)
- Quality
ex:quality
measuresSpeedMeasures Speed(1)
- Tokens Per Second
ex:tokens-per-second
notLookingGoodNot Looking Good(1)
- Smoketest Results
ex:smoketest-results
preservesGradientMagnitudeFor1dOscillatorParamsPreserves Gradient Magnitude For1d Oscillator Params(1)
- Fix Proposal
ex:fix-proposal
previouslySuggestedWorsePerformancePreviously Suggested Worse Performance(1)
- Old 12l Numbers
ex:old-12l-numbers
providesRealOrderParameterToProvides Real Order Parameter to(1)
- Form 2
ex:form-2
questionsMonotonicLinearBehaviorQuestions Monotonic Linear Behavior(1)
- Lisamegawatts
ex:lisamegawatts
relatesToRelates to(1)
- Rotational Adam W
ex:rotational-adam-w
reportsExperimentalResultsReports Experimental Results(1)
- Xenonfun
ex:xenonfun
sharesGpuUsageWithShares Gpu Usage With(1)
- Spsa
ex:spsa
shouldPerformMuchBetterShould Perform Much Better(1)
- Pretrained Checkpoint
ex:pretrained-checkpoint
showsStabilityShows Stability(1)
- Val Loss Trajectory
ex:val-loss-trajectory
signalsPhysicsModeSignals Physics Mode(1)
- R Global
ex:r-global
suspectsNormalizationInterferesWithSignalingSuspects Normalization Interferes With Signaling(1)
- Lisamegawatts
ex:lisamegawatts
trainedWithTrained With(1)
- Experiment Setup
ex:experiment-setup
usedInGatingUsed in Gating(1)
- R Global
ex:r-global
usesDirectlyInUses Directly in(1)
- Fix for Zero Padding
ex:fix-for-zero-padding
usesOptimizerUses Optimizer(1)
- Test 2
ex:test-2
Other facts (135)
The long tail: predicates that appear too rarely to warrant their own section. Filter or scroll to find a specific one. Each row links to its source.
135 facts
| Predicate | Value | Ref |
|---|---|---|
| Rdf:type | Algorithm | [21] |
| Rdf:type | Software System | [22] |
| Rdf:type | Optimizer Class | [23] |
| Rdf:type | Optimization Algorithm | [24] |
| Rdf:type | Optimizer | [25] |
| Rdf:type | Optimizer | [26] |
| Rdf:type | Software Component | [28] |
| Uses | Kan Gates | [1] |
| Uses | Linearized Error Propagation | [1] |
| Uses | Harmonic Decomposition | [1] |
| Uses | Rotational Updates | [1] |
| Uses | Lohe Synchronization | [1] |
| Uses | Linearized Error Propagation | [12] |
| Is Backprop Free | true | [1] |
| Is Backprop Free | true | [12] |
| Has Val Ppl | 1119933.4 | [2] |
| Has Val Ppl | 349.9 | [10] |
| Has Val Loss | 13.9288 | [2] |
| Has Val Loss | 5.8577 | [10] |
| At Step | Step 200 | [10] |
| At Step | Step 100 | [10] |
| Has Metric Value | 0.59 | [21] |
| Has Metric Value | 5.8 | [21] |
| Interpretation | no phase ordering | [21] |
| Interpretation | not learning structure from scratch | [21] |
| Has Built in | Harmonic Decomposition | [1] |
| Integrates Ka Ngates | true | [1] |
| Captures Attention Order Parameters Via | Hooks | [1] |
| Captures Via Hooks | Attention Order Parameters | [1] |
| Is Numbered | 4 | [1] |
| Feeds Back Into | Optimization | [1] |
| Is Not Viable at | this depth/scale | [2] |
| Completely Diverged | true | [2] |
| Has Tr Loss | 13.6893 | [2] |
| Has Tokens Per Sec | 21902 | [2] |
| Uses Backprop | no | [2] |
| Presupposes Mean Field Limit | true | [2] |
| Has Ppl Approx | 1000000 | [2] |
| Has Highest Loss | Experiment Setup | [2] |
| Deemed Not Viable | true | [2] |
| Has Vs Unig | 7.2888 | [2] |
| Uses Gpu Gb | 3 | [2] |
| Has Fastest Tok Per Sec | 21902 | [2] |
| Has Ms Per Step | 181 | [2] |
| Has Tr Ppl | 881394.4 | [2] |
| Achieves Ppl | 27.9 | [3] |
| Exists Currently | current | [3] |
| Relates to | Boltzmann Distribution | [4] |
| Has Internal | Get Temperature Scheduler | [4] |
| With Coupling | 0.2 | [5] |
| Uses Lr Ffn | 0.06 | [5] |
| Causes Rotation | Weight Updates | [5] |
| Depends on | Features Tensor | [5] |
| Is for | Backprop Free Fine Tune | [5] |
| Has Speed Advantage | 17K vs 11.6K tok/s | [6] |
| Experiences No Crash | During Training | [6] |
| Confirms Compatibility With | Lohe V2 Ffn | [6] |
| Has Recommended Use Case | Fine Tune From Pretrained Only | [6] |
| Presupposes Pretrained Models Exist | For Fine Tuning | [6] |
| Expected to Converge | true | [7] |
| References Harmonic Mlx Repo | Github Link | [7] |
| Has Oscillator Parameters | Kappa | [7] |
| Faster Wall Time | 2.2 min | [7] |
| Involves1d Oscillator Params | true | [7] |
| Is Linearized Gradient Method | true | [7] |
| Performs Poorly on | Pretrained Checkpoint | [8] |
| Actively Destroying | Checkpoint | [8] |
| Started From | Val Ppl 417 Pretrained | [8] |
| Is Spiking to | 15-17 | [8] |
| Gates Using | R Global | [9] |
| Uses Gate | 4·r·(1-r) | [9] |
| Returned to | Working State | [10] |
| Has Wall Time S | 54.5 | [10] |
| Has Tok Per S | 15030 | [10] |
| Has Train Loss | 5.909 | [10] |
| Confirmed Working | true | [10] |
| Presupposes Prior Non Working State | Working State | [10] |
| Preferred Optimizer | true | [10] |
| Adapts to Attention Type | Lohe Spherical | [11] |
| Detects Automatically | R Global | [11] |
| Switches to | Physics Gate | [11] |
| Advertised As | Backprop Free Linearized | [12] |
| Is | Optimizer | [12] |
| Syncs Without Backprop | Lohe Sync | [13] |
| Previously Used Hardcoded R | 0.5 | [13] |
| Never Run With | Form 2 Gate Active | [13] |
| Presupposes No Backprop Viability | Lohe Sync | [13] |
| Works on | Finetune | [14] |
| Seemed to Work Only on | Finetune | [14] |
| Fails to Reduce Loss on | Pretraining or Smoketest | [14] |
| Lacks Awareness of | Loss Decrease | [14] |
| Related to Historical | Synchronization Optimizers | [15] |
| Has Forward Only Path | true | [16] |
| Optimizes | Sphere Geometry | [17] |
| Is Compared to | Rot Adam W | [17] |
| Optimizes Directly | Sphere Geometry | [17] |
| Outperforms | Rot Adam W | [17] |
| Rejects Form2 Lohe Attention When | H Not Equals 4 | [18] |
| Avoids Autograd Backward | true | [19] |
| Measured at Scale | Six Layer Scale | [19] |
Timeline
Timeline axis is valid_time — when each source says the fact was true in the world, not when Dontopedia learned about it. Retracted rows are kept for provenance; coloured stripes indicate the context kind.
—
usesblah/watt-activation/part-117
ex:kan-gates
—
hasBuiltInblah/watt-activation/part-117
ex:harmonic-decomposition
—
usesblah/watt-activation/part-117
ex:linearized-error-propagation
—
integratesKANgatesblah/watt-activation/part-117
true
—
capturesAttentionOrderParametersViablah/watt-activation/part-117
ex:hooks
—
capturesViaHooksblah/watt-activation/part-117
ex:attention-order-parameters
—
isNumberedblah/watt-activation/part-117
4
—
isBackpropFreeblah/watt-activation/part-117
true
—
usesblah/watt-activation/part-117
ex:harmonic-decomposition
—
feedsBackIntoblah/watt-activation/part-117
ex:optimization
—
usesblah/watt-activation/part-117
ex:rotational-updates
—
usesblah/watt-activation/part-117
ex:lohe-synchronization
—
isNotViableAtblah/watt-activation/part-121
this depth/scale
—
completelyDivergedblah/watt-activation/part-121
true
—
hasTrLossblah/watt-activation/part-121
13.6893
—
hasTokensPerSecblah/watt-activation/part-121
21902
—
usesBackpropblah/watt-activation/part-121
no
—
presupposesMeanFieldLimitblah/watt-activation/part-121
true
—
hasPplApproxblah/watt-activation/part-121
1000000
—
hasHighestLossblah/watt-activation/part-121
ex:experiment-setup
—
deemedNotViableblah/watt-activation/part-121
true
—
hasVsUnigblah/watt-activation/part-121
7.2888
—
usesGpuGbblah/watt-activation/part-121
3
—
hasFastestTokPerSecblah/watt-activation/part-121
21902
—
hasMsPerStepblah/watt-activation/part-121
181
—
hasValPplblah/watt-activation/part-121
1119933.4
—
hasValLossblah/watt-activation/part-121
13.9288
—
hasTrPplblah/watt-activation/part-121
881394.4
—
achievesPplblah/watt-activation/part-123
27.9
—
existsCurrentlyblah/watt-activation/part-123
current
—
relatesToblah/watt-activation/part-179
ex:boltzmann-distribution
—
hasInternalblah/watt-activation/part-179
ex:_get-temperature-scheduler
—
withCouplingblah/watt-activation/part-191
0.2
—
usesLrFfnblah/watt-activation/part-191
0.06
—
causesRotationblah/watt-activation/part-191
ex:weight-updates
—
dependsOnblah/watt-activation/part-191
ex:features-tensor
—
isForblah/watt-activation/part-191
ex:backprop-free-fine-tune
—
hasSpeedAdvantageblah/watt-activation/part-183
17K vs 11.6K tok/s
—
experiencesNoCrashblah/watt-activation/part-183
ex:during-training
—
confirmsCompatibilityWithblah/watt-activation/part-183
ex:lohe-v2-ffn
—
hasRecommendedUseCaseblah/watt-activation/part-183
ex:fine-tune-from-pretrained-only
—
presupposesPretrainedModelsExistblah/watt-activation/part-183
ex:for-fine-tuning
—
expectedToConvergeblah/watt-activation/part-192
true
—
referencesHarmonicMLXRepoblah/watt-activation/part-192
ex:github-link
—
hasOscillatorParametersblah/watt-activation/part-192
ex:kappa
—
fasterWallTimeblah/watt-activation/part-192
2.2 min
—
involves1dOscillatorParamsblah/watt-activation/part-192
true
—
isLinearizedGradientMethodblah/watt-activation/part-192
true
—
performsPoorlyOnblah/watt-activation/part-190
ex:pretrained-checkpoint
—
activelyDestroyingblah/watt-activation/part-190
ex:checkpoint
—
startedFromblah/watt-activation/part-190
ex:val-ppl-417-pretrained
—
isSpikingToblah/watt-activation/part-190
15-17
—
gatesUsingblah/watt-activation/part-199
ex:r-global
—
usesGateblah/watt-activation/part-199
4·r·(1-r)
—
returnedToblah/watt-activation/part-196
ex:working-state
—
hasWallTimeSblah/watt-activation/part-196
54.5
—
hasTokPerSblah/watt-activation/part-196
15030
—
hasTrainLossblah/watt-activation/part-196
5.909
—
confirmedWorkingblah/watt-activation/part-196
true
—
presupposesPriorNonWorkingStateblah/watt-activation/part-196
ex:working-state
—
preferredOptimizerblah/watt-activation/part-196
true
—
atStepblah/watt-activation/part-196
ex:step-200
—
hasValLossblah/watt-activation/part-196
5.8577
—
atStepblah/watt-activation/part-196
ex:step-100
—
hasValPplblah/watt-activation/part-196
349.9
—
adaptsToAttentionTypeblah/watt-activation/part-201
ex:lohe-spherical
—
detectsAutomaticallyblah/watt-activation/part-201
ex:r-global
—
switchesToblah/watt-activation/part-201
ex:physics-gate
—
isBackpropFreeblah/watt-activation/part-189
true
—
advertisedAsblah/watt-activation/part-189
ex:backprop-free-linearized
—
usesblah/watt-activation/part-189
ex:linearized-error-propagation
—
isblah/watt-activation/part-189
ex:optimizer
—
syncsWithoutBackpropblah/watt-activation/part-205
ex:lohe-sync
—
previouslyUsedHardcodedRblah/watt-activation/part-205
0.5
—
neverRunWithblah/watt-activation/part-205
ex:form-2-gate-active
—
presupposesNoBackpropViabilityblah/watt-activation/part-205
ex:lohe-sync
—
worksOnblah/watt-activation/part-214
ex:finetune
—
seemedToWorkOnlyOnblah/watt-activation/part-214
ex:finetune
—
failsToReduceLossOnblah/watt-activation/part-214
ex:pretraining-or-smoketest
—
lacksAwarenessOfblah/watt-activation/part-214
ex:loss-decrease
—
relatedToHistoricalblah/watt-activation/part-211
ex:synchronization-optimizers
—
hasForwardOnlyPathblah/watt-activation/part-221
true
—
optimizesblah/watt-activation/part-228
ex:sphere-geometry
—
isComparedToblah/watt-activation/part-228
ex:rot-adam-w
—
optimizesDirectlyblah/watt-activation/part-228
ex:sphere-geometry
—
outperformsblah/watt-activation/part-228
ex:rot-adam-w
—
rejectsForm2LoheAttentionWhenblah/watt-activation/part-261
ex:h-not-equals-4
—
avoidsAutogradBackwardblah/watt-activation/part-294
true
—
measuredAtScaleblah/watt-activation/part-294
ex:six-layer-scale
—
implementsJacobianChainblah/watt-activation/part-294
true
—
hasStepBreakdownblah/watt-activation/part-294
ex:lohe-optimizer-step-breakdown
—
hasSlownessPercentageblah/watt-activation/part-294
7
—
testedAtSixLayersblah/watt-activation/part-294
true
—
hasNearIdenticalForwardblah/watt-activation/part-294
ex:rotadamw
—
sharesModelForwardWithblah/watt-activation/part-294
ex:rotadamw
—
scalesLinearlyWithLayersblah/watt-activation/part-294
ex:block-update-cost
—
hasBatchedEvalImprovementblah/watt-activation/part-294
17
—
couldBecomeFasterThanblah/watt-activation/part-294
ex:rotadamw-backward
—
isSlowerThanblah/watt-activation/part-294
ex:rotadamw
—
performsBetterRelativelyAtSmallerScaleblah/watt-activation/part-294
ex:six-layer-scale
—
improvedAfterBatchingblah/watt-activation/part-294
true
—
isBiologicalStyleUpdateRuleblah/watt-activation/part-213
forward-pass-only for fine-tuning
—
notSuitableForblah/watt-activation/part-213
from scratch training
—
isDifferentFromblah/watt-activation/part-213
ex:rot-adam-w
—
typeblah/watt-activation/186
ex:Algorithm
—
labelblah/watt-activation/186
Lohe
—
metricStatusblah/watt-activation/186
flatline across all metrics
—
hasMetricValueblah/watt-activation/186
0.59
—
hasMetricValueblah/watt-activation/186
5.8
—
interpretationblah/watt-activation/186
no phase ordering
—
interpretationblah/watt-activation/186
not learning structure from scratch
—
labelblah/watt-activation/188
LoheOptimizer
—
typeblah/watt-activation/188
ex:SoftwareSystem
—
typeblah/watt-activation/189
ex:OptimizerClass
—
labelblah/watt-activation/189
LoheOptimizer
—
hasDescriptionblah/watt-activation/189
backprop-free, linearized error propagation
—
backpropStatusblah/watt-activation/189
backprop-free
—
errorPropagationMethodblah/watt-activation/189
linearized error propagation
—
typeblah/watt-activation/190
ex:OptimizationAlgorithm
—
labelblah/watt-activation/190
LoheOptimizer
—
hasValidationPplStartblah/watt-activation/190
417
—
spikingToblah/watt-activation/190
15-17
—
hasEffectOnblah/watt-activation/190
ex:pretrained-checkpoint
—
destroyingblah/watt-activation/190
ex:pretrained-checkpoint
—
hasIssueblah/watt-activation/190
ex:spiking-behavior
—
typeblah/watt-activation/210
ex:Optimizer
—
labelblah/watt-activation/210
Lohe optimizer
—
typeblah/watt-activation/220
ex:Optimizer
—
labelblah/watt-activation/220
LoheOptimizer
—
hasPathTypeblah/watt-activation/220
ex:forward-only-path
—
findsblah/watt-activation/227
ex:gelation-attractor
—
findsBeforeblah/watt-activation/227
ex:gradient-based-optimizer
—
typeblah/watt-activation/292
ex:SoftwareComponent
—
labelblah/watt-activation/292
LoheOptimizer
—
totalStepTimeblah/watt-activation/292
492.5
—
hasThroughputblah/watt-activation/292
66500
—
forwardCaptureTimeblah/watt-activation/292
143.4
—
hasNoteblah/watt-activation/292
same as RotAdamW forward
—
blockUpdateTimeblah/watt-activation/292
344.3
—
blockUpdatePercentageblah/watt-activation/292
70
—
gpuSyncBarrierCountblah/watt-activation/292
77
—
r_globalItemTimeblah/watt-activation/292
2.7
References (28)
28 references
ctx:discord/blah/watt-activation/part-117ctx:discord/blah/watt-activation/part-121ctx:discord/blah/watt-activation/part-123ctx:discord/blah/watt-activation/part-179ctx:discord/blah/watt-activation/part-191ctx:discord/blah/watt-activation/part-183ctx:discord/blah/watt-activation/part-192ctx:discord/blah/watt-activation/part-190ctx:discord/blah/watt-activation/part-199ctx:discord/blah/watt-activation/part-196ctx:discord/blah/watt-activation/part-201ctx:discord/blah/watt-activation/part-189ctx:discord/blah/watt-activation/part-205ctx:discord/blah/watt-activation/part-214ctx:discord/blah/watt-activation/part-211ctx:discord/blah/watt-activation/part-221ctx:discord/blah/watt-activation/part-228ctx:discord/blah/watt-activation/part-261ctx:discord/blah/watt-activation/part-294ctx:discord/blah/watt-activation/part-213ctx:discord/blah/watt-activation/186- full textwatt-activation-186text/plain3 KB
doc:agent/watt-activation-186/28dd0cf1-877b-4d3c-8650-c102a40c3031Show excerpt
[2026-03-10 03:00] lisamegawatts: might need to feed it how to compute these [2026-03-10 03:00] xenonfun: `logs/phase_metrics_20260309_225648.jsonl` log collections ``` {"r": 0.6015, "b": 5.6972, "pm": 0.2788, "pv": 0.1787}, {"r": 0.5995, "…
ctx:discord/blah/watt-activation/188- full textwatt-activation-188text/plain3 KB
doc:agent/watt-activation-188/0b24c5f9-ca6d-47b7-9d97-98b6fac36e0cShow excerpt
[2026-03-10 03:16] xenonfun: well I imagine data from working RotAdamW will be informative for it as to how to correct behavior / step issues in LoheOptimizer [2026-03-10 03:17] xenonfun: also that will be recorded [2026-03-10 03:38] xenonf…
ctx:discord/blah/watt-activation/189- full textwatt-activation-189text/plain2 KB
doc:agent/watt-activation-189/ee6e7700-8f8f-458c-bd97-cd00204ffe29Show excerpt
[2026-03-10 03:42] xenonfun: ``` What the fix looks like: Coupling κ_g is a scalar per group. Its gradient through the sync step is tractable: at first order, Δcoupling_g ∝ -(∂loss/∂spectra_synced) · (mean_spec_g - spectra_g) — the reado…
ctx:discord/blah/watt-activation/190- full textwatt-activation-190text/plain3 KB
doc:agent/watt-activation-190/afd2a2d6-55fc-43d9-9655-83cd2755213fShow excerpt
[2026-03-10 03:51] xenonfun: ⏺ The fix is working. Here's the comparison: ``` ┌────────────────┬────────────────┬────────────────────────────┐ │ Metric │ Before fix │ After fix │ ├────────────────┼────────…
ctx:discord/blah/watt-activation/210- full textwatt-activation-210text/plain2 KB
doc:agent/watt-activation-210/20d41130-276e-4126-8cde-1fc6d9275db1Show excerpt
[2026-03-11 03:59] omega [bot]: The spectral and harmonic signatures being equivalent means that the graph spectral decomposition (eigenvalues/eigenvectors of graph Laplacian or adjacency) directly corresponds to the harmonic modes of the s…
ctx:discord/blah/watt-activation/220- full textwatt-activation-220text/plain3 KB
doc:agent/watt-activation-220/5c7f4a28-90e7-46de-ae1e-9e19a58c8d65Show excerpt
[2026-03-11 04:42] xenonfun: FFN DFT — much richer specialization than spectral: ``` ┌─────┬───────┬────────┬────────────────┬────────────────┐ │ blk │ r │ FFN DC │ dominant mode │ pattern │ ├─────┼───────┼────────┼───────…
ctx:discord/blah/watt-activation/227- full textwatt-activation-227text/plain2 KB
doc:agent/watt-activation-227/68739c39-b81e-46d6-a743-64f53964d850Show excerpt
[2026-03-11 05:22] xenonfun: ⏺ The data shows it clearly. The gelation onset is between step 100 and step 200: step loss b9_r b10_r b11_r b10−b11 b9_disorder 100 11.80 0.363 0.347 0.362 −0.014 −0.016 ← symmetric, …
ctx:discord/blah/watt-activation/292- full textwatt-activation-292text/plain2 KB
doc:agent/watt-activation-292/126da404-b457-4703-9947-909873e9c37aShow excerpt
[2026-03-14 03:52] xenonfun: ``` ⏺ There it is: LOHE OPTIMIZER STEP BREAKDOWN (492.5 ms total, 66.5K tok/s) ├── Forward + capture: 143.4 ms (29%) — same as RotAdamW forward ├── Block updates: 344.3 ms (70%) ← THE …
See also
- Kan Gates
- Harmonic Decomposition
- Linearized Error Propagation
- Hooks
- Attention Order Parameters
- Optimization
- Rotational Updates
- Lohe Synchronization
- Experiment Setup
- Boltzmann Distribution
- Get Temperature Scheduler
- Weight Updates
- Features Tensor
- Backprop Free Fine Tune
- During Training
- Lohe V2 Ffn
- Fine Tune From Pretrained Only
- For Fine Tuning
- Github Link
- Kappa
- Pretrained Checkpoint
- Checkpoint
- Val Ppl 417 Pretrained
- R Global
- Working State
- Step 200
- Step 100
- Lohe Spherical
- Physics Gate
- Backprop Free Linearized
- Optimizer
- Lohe Sync
- Form 2 Gate Active
- Finetune
- Pretraining or Smoketest
- Loss Decrease
- Synchronization Optimizers
- Sphere Geometry
- Rot Adam W
- H Not Equals 4
- Six Layer Scale
- Lohe Optimizer Step Breakdown
- Rotadamw
- Block Update Cost
- Rotadamw Backward
- Algorithm
- Software System
- Optimizer Class
- Optimization Algorithm
- Spiking Behavior
- Optimizer
- Forward Only Path
- Gelation Attractor
- Gradient Based Optimizer
- Software Component
Keep researching
Missing something or suspicious of what's here? Kick off a research session — a Claude agent will investigate, cite its sources, and file new facts into a dedicated context you can review before accepting into the shared view.