Lohe Spherical Attention

as donto believed on

From Dontopedia, the open, paraconsistent wiki. (Last updated 2026-06-06.)

Has Parameter	H=4^[1]
Has Parameter	n_layers=4^[1]
Outputs	Attention Values^[9]
Outputs	Sync State^[9]
Projects	K^[9]
Projects	Q^[9]
Parameter Name	n_ffn_harmonics^[12]
Parameter Name	n_ffn_groups^[12]
Had Prior Ppl	1350^[8]
Has Best Ppl Times Tok S Efficiency at Rot0 1	fast + good quality^[8]
Has Loss at Rot0 1	6.37^[8]
Has Params at Rot0 1	12.3M^[8]

ex:lohe-spherical-attention

Lohe Spherical Attention has 86 facts recorded in Dontopedia across 13 references, with 4 live disagreements.

86 facts·76 predicates·13 sources·4 in dispute

Mostly:has parameter(7), parameter name(3), outputs(2)

Has Parameterin dispute`hasParameter`

H=4^[1]all time · Part 201
n_layers=4^[1]all time · Part 201
batch=4^[1]all time · Part 201
d_model=256^[1]all time · Part 201
vocab=1024^[1]all time · Part 201
seq_len=128^[1]all time · Part 201
G=4^[1]all time · Part 201

Outputsin dispute`outputs`

Attention Values^[9]all time · Part 197
Sync State^[9]all time · Part 197

Projectsin dispute`projects`

K^[9]all time · Part 197
Q^[9]all time · Part 197

Parameter Namein dispute`parameterName`

n_ffn_harmonics^[12]all time · 258
n_ffn_groups^[12]all time · 258
heads^[12]all time · 258

Has Best Ppl Times Tok S Efficiency at Rot0 1`hasBestPPLTimesTokSEfficiencyAtRot0-1`

fast + good quality^[8]all time · Part 318

Has Loss at Rot0 1`hasLossAtRot0-1`

6.37^[8]all time · Part 318

Has Params at Rot0 1`hasParamsAtRot0-1`

12.3M^[8]all time · Part 318

Has Ppl at Rot0 1`hasPPLAtRot0-1`

582^[8]all time · Part 318

Has Tok Per S at Rot0 1`hasTokPerSAtRot0-1`

83625^[8]all time · Part 318

Is3 2x Faster Than Topo at Rot0 1`is3-2xFasterThanTopoAtRot0-1`

Topo Star Kan Attention^[8]all time · Part 318

Is47 Percent Faster Than Spectral At0 1`is47PercentFasterThanSpectralAt0-1`

Spectral Attention^[8]all time · Part 318

Inbound mentions (20)

Other subjects in dontopedia point AT this entity as a value. These are inverse relationships — e.g. "X motherOf this subject" — and answer questions the forward facts can't. Grouped by predicate.

`advocatesUsage`Advocates Usage(1)

Xenonfunex:xenonfun

`attentionType`Attention Type(1)

Lohe Spherical Attention Rot 0 1 Resultex:lohe-spherical-attention-rot-0-1-result

`composedOf`Composed of(1)

Block Stackex:block-stack

`concernsImplementation`Concerns Implementation(1)

Lohe Spherical Attention Smoke Testex:lohe-spherical-attention-smoke-test

`discussesLoheSphericalAttention`Discusses Lohe Spherical Attention(1)

Xenonfunex:xenonfun

`focusesOnModelTraining`Focuses on Model Training(1)

Conversationex:conversation

`hasAttentionType`Has Attention Type(1)

Session 2026 03 21ex:session-2026-03-21

`hasNaturalPairedAttention`Has Natural Paired Attention(1)

Lohe Ffnv2ex:lohe-ffnv2

`hasParameter`Has Parameter(1)

Full Principled Stackex:full-principled-stack

`hasSubcomponent`Has Subcomponent(1)

Harmonic Block 1ex:harmonic-block-1

`implementsAttentionType`Implements Attention Type(1)

Rust Projectex:rust-project

`involvesSketching`Involves Sketching(1)

Option Twoex:option-two

`isMorePureVersion`Is More Pure Version(1)

Lohespherical Attentionex:lohespherical-attention

`isPurer`Is Purer(1)

Lohe Spherical Attentionex:lohe-spherical-attention

`isRunning`Is Running(1)

Xenonfunex:xenonfun

`requiresRewrite`Requires Rewrite(1)

Ar Decoderex:ar-decoder

`usesAttention`Uses Attention(1)

Multimodal Harmonicmlxex:multimodal-harmonicmlx

`verifiesBasicFunctionality`Verifies Basic Functionality(1)

Smoke Testex:smoke-test

`wasCompletelyDanglingIn`Was Completely Dangling in(1)

Harmonic Coeffsex:harmonic-coeffs

`wasNotForwardingInto`Was Not Forwarding Into(1)

Harmonic Blockex:harmonic-block

Other facts (64)

The long tail: predicates that appear too rarely to warrant their own section. Filter or scroll to find a specific one. Each row links to its source.

64 facts

Predicate	Value	Ref
Overtakes at Higher Rot	Topo Star Kan Attention	[8]
Overtakes Topo Star Kan at Rot0 1	Topo Star Kan Attention	[8]
Uses27 Percent Fewer Params Than Spectral	Spectral Attention	[8]
Within27 Percent Ppl of Spectral At0 1	Spectral Attention	[8]
Does Not Use Heads	Heads	[7]
Has Nffn Groups	N Ffn Groups	[7]
Has Nffn Harmonics	N Ffn Harmonics	[7]
Does Not Hard Ban Non4 H in Forward Inference	null	[6]
Uses Exact Quaternion Sync at	H Equals 4	[6]
Uses Generic Spectral Sync Otherwise	null	[6]
Applied to	Multimodal Harmonicmlx	[2]
Described in Attachment	Screenshot 2026 03 11 at 10 17 19 Pm Png	[4]
Has Live Visualization	Lohe Sph Live2 Png	[4]
Exists As Option	Attn Type	[1]
Has Loss Descent	0.176	[1]
Has Lower Throughput	Spectral Plus Lohe V2	[1]
Has Mean R Global During Training	0.61	[1]
Has R Global at Random Init	0.69	[1]
Has Throughput	29.5K tok/s	[1]
Is Attention Type	Model Config	[1]
Is Small Model Config	G=4, H=4	[1]
Is Tested in	Smoke Test	[1]
Is Usable Via	attn_type="lohe_spherical"	[1]
Maintains R Global Stability	Valid Range	[1]
Matches Loss Descent	Spectral Plus Lohe V2	[1]
Documented in	Docs Claude Lohe Attention Md	[5]
Has Fewer Attention Params	48%	[5]
Has Same Dynamics As	Ffnv2	[5]
Has Throughput Cost	~10% from sync step	[5]
Planned Implementation	{}	[5]
Reduces Params by	48%	[5]
Eliminates Need for	R Equals 0 5 Proxy	[9]
Is Natural Pairing	Lohe Ffnv2	[9]
Outputs R for	Optimizer Gate	[9]
Projects Onto	S H Minus 1	[9]
Runs	Spectral Lohe Sync	[9]
Dominates Efficiency Frontier at Rot0 1	Spectral Attention	[8]
Is Component	null	[11]
Has Default Geometry	8x4	[6]
Uses Models Requested Geometry	null	[6]
Exists As Pure Variant	true	[10]
Is Purer	Lohe Spherical Attention	[10]
Achieves Sync	Within Block	[1]
Contains R Global	Attention Infos	[1]
Dominates by	Proj V Proj Out	[1]
Exhibits Within Block Sync	Already Present	[1]
Expects Smaller Throughput Gap	Gpt 2 Scale	[1]
Is Viable Alternative	Spectral Attention	[1]
Provides R Global Metric	Attention Infos	[1]
Scales to Gpt2	Gpt 2	[1]
Has Parameter Efficiency	using 27% fewer params	[3]
Has Speed Advantage	47% faster	[3]
Compared to	Spectral Attention	[3]
Has Perplexity Proximity	within 27% PPL	[3]
Has Tokens Per Second	83625	[3]
Has Speed Multiplier	3.2× faster	[3]
Has Perplexity	582	[3]
Overtakes Ranking of	Topo Star Plus Kan Attention	[3]
Has Current Perplexity	582	[3]
Has Previous Perplexity	1350	[3]
Perplexity Change	57% better	[3]
Rdf:type	Attention Type	[3]
Rdfs:label	lohe_spherical	[3]
Related to	Lohe Ffn V1	[13]

Timeline axis is valid_time — when each source says the fact was true in the world, not when Dontopedia learned about it. Retracted rows are kept for provenance; coloured stripes indicate the context kind.

86 undated

—

achievesSyncblah/watt-activation/part-201

ex:within-block

—

appliedToblah/watt-activation/part-240

ex:multimodal-harmonicmlx

—

comparedToblah/watt-activation/316

ex:spectral-attention

—

containsRGlobalblah/watt-activation/part-201

ex:attention-infos

—

describedInAttachmentblah/watt-activation/part-239

ex:screenshot-2026-03-11-at-10-17-19-pm-png

—

documentedInblah/watt-activation/part-199

ex:docs-claude-lohe-attention-md

—

doesNotHardBanNon4HInForwardInferenceblah/watt-activation/part-261

null

—

doesNotUseHeadsblah/watt-activation/part-260

ex:heads

—

dominatesByblah/watt-activation/part-201

ex:proj-v-proj-out

—

dominatesEfficiencyFrontierAtRot0-1blah/watt-activation/part-318

ex:spectral-attention

—

eliminatesNeedForblah/watt-activation/part-197

ex:r-equals-0-5-proxy

—

exhibitsWithinBlockSyncblah/watt-activation/part-201

ex:already-present

—

existsAsOptionblah/watt-activation/part-201

ex:attn-type

—

existsAsPureVariantblah/watt-activation/part-238

true

—

expectsSmallerThroughputGapblah/watt-activation/part-201

ex:gpt-2-scale

—

hadPriorPPLblah/watt-activation/part-318

1350

—

hasBestPPLTimesTokSEfficiencyAtRot0-1blah/watt-activation/part-318

fast + good quality

—

hasCurrentPerplexityblah/watt-activation/316

582

—

hasDefaultGeometryblah/watt-activation/part-261

ex:8x4

—

hasFewerAttentionParamsblah/watt-activation/part-199

48%

—

hasLiveVisualizationblah/watt-activation/part-239

ex:lohe-sph-live2-png

—

hasLossAtRot0-1blah/watt-activation/part-318

6.37

—

hasLossDescentblah/watt-activation/part-201

0.176

—

hasLowerThroughputblah/watt-activation/part-201

ex:spectral-plus-lohe-v2

—

hasMeanRGlobalDuringTrainingblah/watt-activation/part-201

0.61

—

hasNffnGroupsblah/watt-activation/part-260

ex:n-ffn-groups

—

hasNffnHarmonicsblah/watt-activation/part-260

ex:n-ffn-harmonics

—