Dontopedia
Explore

Spectral Attention

From Dontopedia, the open, paraconsistent wiki. (Last updated 2026-06-06.)

Spectral Attention has 64 facts recorded in Dontopedia across 17 references, with 6 live disagreements.

64 facts·50 predicates·17 sources·6 in dispute

Mostly:rdfs:label(6), rdf:type(5), originally had separate projs(3)

Maturity scale raw canonical shape-checked rule-derived certified

Uses Funk Hecke Harmonic Decompositionin disputeusesFunkHeckeHarmonicDecomposition

Lacksin disputelacks

Is Member ofin disputeisMemberOf

Originally Had Separate Projsin disputeoriginallyHadSeparateProjs

Rdfs:labelin disputerdfs:label

  • Spectral attention[15]all time · 218
  • SpectralAttention[9]all time · 158
  • spectral[10]sourceall time · 103
  • SpectralAttention[16]all time · 118
  • SpectralAttention[3]sourceall time · 105
  • spectral[4]sourceall time · 316

Rdf:typein disputerdf:type

Had Prior PplhadPriorPPL

  • 754[6]all time · Part 318

Has Best Quality Moderate Speed at Rot0 1hasBestQualityModerateSpeedAtRot0-1

  • PPL 457, Tok/s 56,736[6]all time · Part 318

Has Loss at Rot0 1hasLossAtRot0-1

  • 6.12[6]all time · Part 318

Has Params at Rot0 1hasParamsAtRot0-1

  • 16.9M[6]all time · Part 318

Has Ppl at Rot0 1hasPPLAtRot0-1

  • 457[6]all time · Part 318

Has Tok Per S at Rot0 1hasTokPerSAtRot0-1

  • 56736[6]all time · Part 318

Inbound mentions (15)

Other subjects in dontopedia point AT this entity as a value. These are inverse relationships — e.g. "X motherOf this subject" — and answer questions the forward facts can't. Grouped by predicate.

attentionTypeAttention Type(1)

comparedToCompared to(1)

dominatesEfficiencyFrontierAtRot0-1Dominates Efficiency Frontier at Rot0 1(1)

hasImplementedHas Implemented(1)

includeSpectralAttentionInclude Spectral Attention(1)

includesSpectralAttentionIncludes Spectral Attention(1)

is47PercentFasterThanSpectralAt0-1Is47 Percent Faster Than Spectral At0 1(1)

isCurrentIs Current(1)

isViableAlternativeIs Viable Alternative(1)

matchesSpectralAttentionSizeMatches Spectral Attention Size(1)

mentionsTopicMentions Topic(1)

specifiesContextSpecifies Context(1)

uses27PercentFewerParamsThanSpectralUses27 Percent Fewer Params Than Spectral(1)

wasRelativeToWas Relative to(1)

within27PercentPPLOfSpectralAt0-1Within27 Percent Ppl of Spectral At0 1(1)

Other facts (38)

The long tail: predicates that appear too rarely to warrant their own section. Filter or scroll to find a specific one. Each row links to its source.

38 facts
PredicateValueRef
Prioritizes Quality Over Speedbest quality, moderate speed[6]
Wins on Quality at Both Strengthsrot=0.01 and 0.1[6]
Not Yet Ported FromPython Harmonicmlx[12]
References Funk HeckeFunk Hecke[12]
Gets There More Completelynull[5]
Matches Ffn Frequency Basisnull[5]
Operates inSame Frequency Basis As Ffn[5]
Operates in Same Frequency Basis AsFfn[5]
Has Equivalent Tensor Shape(B, T, 12, 8, 64)[8]
Serves As Baselinenull[8]
Splits V Into Per Head Slicesd_h=64[8]
ReturnsKv State[14]
Is Spectral But NotKuramoto[13]
Requires Calibration tor=0.5[13]
UsesSpectral Basis[13]
Has Clean State HandlingOne Kv State Tensor[7]
Avoids Kv Cachetrue[1]
Implements Fused Qkv ProjectionQkv Proj[11]
Not to Be Replaced bySoftmax[11]
Is Attention Varianttrue[12]
Contextually ComparedFfn Ring Modes[2]
Is Inferiordue to calibration[13]
Lacks CouplingKuramoto[13]
Evaluated at Strengthboth strengths[4]
Wins on Qualitytrue[4]
Has Current Perplexity457[4]
Has Previous Perplexity754[4]
Perplexity Change39% better[4]
Operates in BasisSame Frequency Basis As Ffn[15]
Reason for SuccessResonant Pathway[15]
Reaches StateDC Kuramoto State[15]
Has StateKv State Tensor[9]
Has MethodForward Method[9]
DescriptionFunk-Hecke harmonic decomposition[3]
Has Quality RankingBest Quality Option[10]
Quality Adjusted Tokens Per Second8008[10]
Tokens Per Second210000[10]
Perplexity Score6.03[10]

Timeline

Timeline axis is valid_time — when each source says the fact was true in the world, not when Dontopedia learned about it. Retracted rows are kept for provenance; coloured stripes indicate the context kind.

64 undated
—
avoidsKvCacheblah/watt-activation/part-126
true
—
contextuallyComparedblah/watt-activation/part-216
ex:ffn-ring-modes
—
descriptionblah/watt-activation/105
Funk-Hecke harmonic decomposition
—
evaluatedAtStrengthblah/watt-activation/316
both strengths
—
getsThereMoreCompletelyblah/watt-activation/part-219
null
—
hadPriorPPLblah/watt-activation/part-318
754
—
hasBestQualityModerateSpeedAtRot0-1blah/watt-activation/part-318
PPL 457, Tok/s 56,736
—
hasCleanStateHandlingblah/watt-activation/part-158
ex:one-kv-state-tensor
—
hasCurrentPerplexityblah/watt-activation/316
457
—
hasEquivalentTensorShapeblah/watt-activation/part-203
(B, T, 12, 8, 64)
—
hasLossAtRot0-1blah/watt-activation/part-318
6.12
—
hasMethodblah/watt-activation/158
ex:forward-method
—
hasParamsAtRot0-1blah/watt-activation/part-318
16.9M
—
hasPPLAtRot0-1blah/watt-activation/part-318
457
—
hasPreviousPerplexityblah/watt-activation/316
754
—
hasQualityRankingblah/watt-activation/103
ex:best-quality-option
—
hasStateblah/watt-activation/158
ex:kv-state-tensor
—
hasTokPerSAtRot0-1blah/watt-activation/part-318
56736
—
implementsFusedQkvProjectionblah/watt-activation/part-118
ex:qkv-proj
—
isAttentionVariantblah/watt-activation/part-460
true
—
isInferiorblah/watt-activation/part-197
due to calibration
—
isMemberOfblah/watt-activation/part-460
ex:attention-variants-subsection
—
isMemberOfblah/watt-activation/part-460
ex:not-yet-ported-section
—
isSpectralButNotblah/watt-activation/part-197
ex:kuramoto
—
lacksblah/watt-activation/part-195
ex:gates-field
—
lacksblah/watt-activation/part-197
ex:oscillator-coupling
—
lacksCouplingblah/watt-activation/part-197
ex:kuramoto
—
matchesFFNFrequencyBasisblah/watt-activation/part-219
null
—
notToBeReplacedByblah/watt-activation/part-118
ex:softmax
—
notYetPortedFromblah/watt-activation/part-460
ex:python-harmonicmlx
—
operatesInblah/watt-activation/part-219
ex:same-frequency-basis-as-ffn
—
operatesInBasisblah/watt-activation/218
ex:same-frequency-basis-as-ffn
—
operatesInSameFrequencyBasisAsblah/watt-activation/part-219
ex:ffn
—
originallyHadSeparateProjsblah/watt-activation/part-118
ex:k-proj
—
originallyHadSeparateProjsblah/watt-activation/part-118
ex:q-proj
—
originallyHadSeparateProjsblah/watt-activation/part-118
ex:v-proj
—
perplexityChangeblah/watt-activation/316
39% better
—
perplexityScoreblah/watt-activation/103
6.03
—
prioritizesQualityOverSpeedblah/watt-activation/part-318
best quality, moderate speed
—
qualityAdjustedTokensPerSecondblah/watt-activation/103
8008
—
labelblah/watt-activation/218
Spectral attention
—
labelblah/watt-activation/158
SpectralAttention
—
labelblah/watt-activation/103
spectral
—
labelblah/watt-activation/118
SpectralAttention
—
labelblah/watt-activation/105
SpectralAttention
—
labelblah/watt-activation/316
spectral
—
typeblah/watt-activation/158
ex:AttentionClass
—
typeblah/watt-activation/103
ex:AttentionMechanism
—
typeblah/watt-activation/316
ex:AttentionType
—
typeblah/watt-activation/105
ex:AttentionVariant
—
typeblah/watt-activation/118
ex:CodeComponent
—
reachesStateblah/watt-activation/218
ex:dc-kuramoto-state
—
reasonForSuccessblah/watt-activation/218
ex:resonant-pathway
—
referencesFunkHeckeblah/watt-activation/part-460
ex:funk-hecke
—
requiresCalibrationToblah/watt-activation/part-197
r=0.5
—
returnsblah/watt-activation/part-195
ex:kv-state
—
servesAsBaselineblah/watt-activation/part-203
null
—
splitsVIntoPerHeadSlicesblah/watt-activation/part-203
d_h=64
—
tokensPerSecondblah/watt-activation/103
210000
—
usesblah/watt-activation/part-197
ex:spectral-basis
—
usesFunkHeckeHarmonicDecompositionblah/watt-activation/part-105
ex:funk-hecke
—
usesFunkHeckeHarmonicDecompositionblah/watt-activation/part-460
true
—
winsOnQualityblah/watt-activation/316
true
—
winsOnQualityAtBothStrengthsblah/watt-activation/part-318
rot=0.01 and 0.1

References (17)

17 references
  1. [1]Part 1261 fact
    customctx:discord/blah/watt-activation/part-126
  2. [2]Part 2161 fact
    customctx:discord/blah/watt-activation/part-216
  3. [3]watt-activation-1053 facts
    customctx:discord/blah/watt-activation/105
    • full textwatt-activation-105
      text/plain3 KBdoc:agent/watt-activation-105/561920dc-7f65-4ab4-80fa-8e3162aa9046
      Show excerpt
      [2026-03-08 19:26] xenonfun: ``` What They're Leaving on the Table 1. No mx.compile — Their benchmark and model run eagerly. From our experience with AnchorKAN at similar scale, compiled step gives ~1.5-2x throughput improvement on M…
  4. [4]watt-activation-3167 facts
    customctx:discord/blah/watt-activation/316
    • full textwatt-activation-316
      text/plain3 KBdoc:agent/watt-activation-316/c4f50044-9de8-4f34-94c9-40d82203b46b
      Show excerpt
      [2026-03-15 01:57] xenonfun: ``` rot_strength = 0.1 (new default) ┌────────────────┬──────┬─────┬────────┬────────┐ │ Attention │ Loss │ PPL │ Tok/s │ Params │ ├────────────────┼──────┼─────┼────────┼────────┤ │ spectral …
  5. [5]Part 2194 facts
    customctx:discord/blah/watt-activation/part-219
  6. [6]Part 3188 facts
    customctx:discord/blah/watt-activation/part-318
  7. [7]Part 1581 fact
    customctx:discord/blah/watt-activation/part-158
  8. [8]Part 2033 facts
    customctx:discord/blah/watt-activation/part-203
  9. [9]watt-activation-1584 facts
    customctx:discord/blah/watt-activation/158
    • full textwatt-activation-158
      text/plain2 KBdoc:agent/watt-activation-158/746cbfb1-750c-4df5-ae90-78f0235bc1e9
      Show excerpt
      [2026-03-09 16:27] xenonfun: ⏺ For batch prefill, each attention class's forward() currently runs _gated_cumsum over the full sequence and returns output — but throws away the final recurrent state. The step() method maintains that state …
  10. [10]watt-activation-1036 facts
    customctx:discord/blah/watt-activation/103
    • full textwatt-activation-103
      text/plain3 KBdoc:agent/watt-activation-103/6d322edd-8b82-4859-be6f-bc7033a53fe1
      Show excerpt
      [2026-03-08 18:36] xenonfun: It appears your agents have actually already done all this work <@1211062099137265723> in your repo already. https://github.com/MonumentalSystems/harmonic-gpt/blob/master/docs/M3_DEPLOY_NOTES.md (files: Screens…
  11. [11]Part 1185 facts
    customctx:discord/blah/watt-activation/part-118
  12. [12]Part 4606 facts
    customctx:discord/blah/watt-activation/part-460
  13. [13]Part 1976 facts
    customctx:discord/blah/watt-activation/part-197
  14. [14]Part 1952 facts
    customctx:discord/blah/watt-activation/part-195
  15. [15]watt-activation-2184 facts
    customctx:discord/blah/watt-activation/218
    • full textwatt-activation-218
      text/plain2 KBdoc:agent/watt-activation-218/8c31d20c-8693-4814-8ac3-e83806d6ce35
      Show excerpt
      [2026-03-11 04:40] xenonfun: --- Run 2: lohe_spherical+lohe_v3+RotAdamW (1000 steps) Network-level: global_r≈0.107, β≈25, β_gate flat at 0.119. Notably flatter than spectral. ``` ┌─────┬───────────────────┬───────────┬─────────────┬─…
  16. [16]watt-activation-1182 facts
    customctx:discord/blah/watt-activation/118
    • full textwatt-activation-118
      text/plain3 KBdoc:agent/watt-activation-118/ed79098d-1144-44f5-9941-e6b2b9c1caa7
      Show excerpt
      [2026-03-08 23:43] xenonfun: Code Changes (3 important patterns) 1. Fused QKV projection in SpectralAttention - Separate q_proj, k_proj, v_proj → single qkv_proj = Linear(d_model, 3 * d_model). One matmul instead of three. We should po…
  17. [17]Part 1051 fact
    ctx:discord/blah/watt-activation/part-105

See also

Keep researching

Missing something or suspicious of what's here? Kick off a research session — a Claude agent will investigate, cite its sources, and file new facts into a dedicated context you can review before accepting into the shared view.

↑