spectral
From Dontopedia, the open, paraconsistent wiki. (Last updated 2026-07-03.)
spectral has 106 facts recorded in Dontopedia across 13 references, with 6 live disagreements.
Mostly:has ppl at step(7), rdf:type(6), has tokens per second(2)
Maturity scale
raw canonical shape-checked rule-derived certifiedInbound mentions (53)
Other subjects in dontopedia point AT this entity as a value. These are inverse relationships — e.g. "X motherOf this subject" — and answer questions the forward facts can't. Grouped by predicate.
comparedToCompared to(2)
- Ffn Dft
ex:ffn-dft - Topo Anchor Kan Quality Advantage
ex:topo-anchor-kan-quality-advantage
hasFewerParametersThanHas Fewer Parameters Than(2)
- Anchor Kan
ex:anchor_kan - Lohe Spherical Speed Advantage
ex:lohe-spherical-speed-advantage
believesSpectralBetterLmBelieves Spectral Better Lm(1)
- Xenonfun
ex:xenonfun
comparesPplCompares Ppl(1)
- Row 1
ex:row-1
comparesTokPerSCompares Tok Per S(1)
- Row 2
ex:row-2
containsModelContains Model(1)
- Code Block
ex:code-block
describesForwardThroughputDescribes Forward Throughput(1)
- Lisamegawatts Message 18 58
ex:lisamegawatts-message-18-58
evaluatesAsBetterLanguageModelEvaluates As Better Language Model(1)
- Key Takeaway
ex:key-takeaway
exhibitsMuchRicherSpecializationThanExhibits Much Richer Specialization Than(1)
- Ffn Dft
ex:ffn-dft
expertTypeExpert Type(1)
- Latentmoe
ex:latentmoe
fasterThanFaster Than(1)
- Lohe Spherical Speed Advantage
ex:lohe-spherical-speed-advantage
followsFollows(1)
- Anchor Kan
ex:anchor-kan
has13PercentFewerParamsThanHas13 Percent Fewer Params Than(1)
- Lohe Spherical
ex:lohe-spherical
has19PercentSpeedAdvantageHas19 Percent Speed Advantage(1)
- Anchor Kan
ex:anchor-kan
has27PercentFewerParamsThanHas27 Percent Fewer Params Than(1)
- Lohe Spherical
ex:lohe-spherical
hasHighestRInHas Highest R in(1)
- Block L2
ex:block-l2
hasMemberHas Member(1)
- Code Block
ex:code-block
hasRicherSpecializationHas Richer Specialization(1)
- Ffn Dft
ex:ffn-dft
hasStrongDcSyncHubHas Strong DC Sync Hub(1)
- Row 4
ex:row-4
hasSuperiorQualityHas Superior Quality(1)
- Topo Anchor Kan
ex:topo-anchor-kan
hasSyncHubRoleForConfigHas Sync Hub Role for Config(1)
- Block 10
ex:block-10
hasWeakerSyncHas Weaker Sync(1)
- Lohe Spherical
ex:lohe-spherical
haveParamCountsNearbyHave Param Counts Nearby(1)
- Other Attention Types
ex:other-attention-types
improvesSpeedImproves Speed(1)
- Parallel Prefill
ex:parallel-prefill
includesModelIncludes Model(1)
- Test Execution Plan
ex:test-execution-plan
involvesComponentInvolves Component(1)
- Step Compare Form1
ex:step-compare-form1
is120xFasterIs120x Faster(1)
- Parallel Prefill
ex:parallel-prefill
is25PercentFasterThanIs25 Percent Faster Than(1)
- Lohe Spherical
ex:lohe-spherical
is47PercentFasterThanIs47 Percent Faster Than(1)
- Lohe Spherical
ex:lohe-spherical
isConsistentWithBetterPplIs Consistent With Better Ppl(1)
- Row 5
ex:row-5
isFasterIs Faster(1)
- Anchor Kan
ex:anchor-kan
isHigherInIs Higher in(1)
- Thermodynamic Beta
ex:thermodynamic-beta
isMoreOrderedAtSameCouplingIs More Ordered at Same Coupling(1)
- Row 5
ex:row-5
isNotablyFlatterThanIs Notably Flatter Than(1)
- Beta Gate of Run 2
ex:beta-gate-of-run-2
isOutputOfIs Output of(1)
- Spectral Output Sample
ex:spectral-output-sample
isPatternOfIs Pattern of(1)
- Word Like Runs
ex:word-like-runs
isSparserIs Sparser(1)
- Anchor Kan
ex:anchor-kan
isStrictlyBetterThanIs Strictly Better Than(1)
- Lohe Spherical
ex:lohe-spherical
measuredForMeasured for(1)
- Spectral Throughput Measurement
ex:spectral-throughput-measurement
measuresDifferenceBetweenMeasures Difference Between(1)
- Perplexity Gap
ex:perplexity-gap
mentionsMentions(1)
- Source Utterance
ex:source-utterance
overtakesOvertakes(1)
- Lohe Spherical
ex:lohe-spherical
propertyProperty(1)
- Audio Visual Representations
ex:audio-visual-representations
reportsConvergenceReports Convergence(1)
- Row 1
ex:row-1
reportsHigherThermodynamicBetaReports Higher Thermodynamic Beta(1)
- Row 5
ex:row-5
reportsSysRReports Sys R(1)
- Row 2
ex:row-2
representsSameSignalRepresents Same Signal(1)
- Anchor Kan
ex:anchor-kan
risesInRises in(1)
- Sys R
ex:sys_r
slowerDueTo5DTensorsSlower Due To5 D Tensors(1)
- Anchorkan
ex:anchorkan
tendedTowardByTended Toward by(1)
- Run on Noun Phrases
ex:run-on-noun-phrases
underperformsUnderperforms(1)
- Lohe Spherical
ex:lohe-spherical
Other facts (101)
The long tail: predicates that appear too rarely to warrant their own section. Filter or scroll to find a specific one. Each row links to its source.
| Predicate | Value | Ref |
|---|---|---|
| Has Ppl at Step | 1140 | [8] |
| Has Ppl at Step | 561 | [8] |
| Has Ppl at Step | 501 | [8] |
| Has Ppl at Step | 373 | [8] |
| Has Ppl at Step | 260 | [8] |
| Has Ppl at Step | 219 | [8] |
| Has Ppl at Step | 239 | [8] |
| Rdf:type | Model Component | [9] |
| Rdf:type | Attention Configuration | [10] |
| Rdf:type | Model Architecture | [11] |
| Rdf:type | Attention Type | [12] |
| Rdf:type | Model Kind | [13] |
| Rdf:type | Model | [13] |
| Has Tokens Per Second | 12815 | [6] |
| Has Tokens Per Second | 12815 | [12] |
| Has Loss | 9.02 | [6] |
| Has Loss | 9.02 | [12] |
| Has Param Count | 107.2M | [6] |
| Has Param Count | 107200000 | [12] |
| Achieved Throughput | 43700 | [11] |
| Achieved Throughput | 46300 | [11] |
| Throughput Unit | tok/s | [13] |
| Throughput Unit | tokens per second | [13] |
| Better in Training Than | Pytorch Mps | [1] |
| Peaks at Forward Throughput | 365000 | [1] |
| Is Four Times Faster Than | Pytorch Mps | [1] |
| Has Training Throughput | 84800 | [1] |
| Has Latency Scaling | 7.9 | [1] |
| Exhibits Scaling | Super Linear Ish | [1] |
| Presupposes Competitor to Gpt | {} | [2] |
| Has Essential Stability | Memory Standard | [2] |
| Outperforms | Gpt | [2] |
| Was Stable | Memory Standard | [2] |
| Reaches Same DC Sync Strength As | Anchor Kan | [3] |
| Has Tok S | 12000 | [3] |
| Has R Hub | 0.56 | [3] |
| Has Sync Hub | Block 10 | [3] |
| Sync Hub Precedes | Anchor Kan | [3] |
| Has DC Mode | 0.315 | [3] |
| Tends Toward Run on Noun Phrases | true | [4] |
| Lacks5 D Tensor | true | [4] |
| Faster Than Anchorkan | Anchorkan | [4] |
| References in Comparison | Anchorkan | [4] |
| Outputs Longer Word Like Runs | "The not used in the be 2001 3 -5 of a same" | [4] |
| Has Speed | ~400 tok/s | [4] |
| Includes Numbers and Partial Structure | true | [4] |
| Has Dominant H0 in L2 | true | [5] |
| Forms Sync Hub | L2 | [5] |
| Has26 Percent Ppl Gap | Anchor Kan | [5] |
| Has DC Synchronization Hub | L2 H0 | [5] |
| Is More Ordered | true | [5] |
| Reaches Higher Final Order | Anchor Kan | [5] |
| Has Smoother Convergence | Anchor Kan | [5] |
| Has Higher Beta | Anchor Kan | [5] |
| Optimizes Ppl | true | [5] |
| Implements True Rope in Qk | true | [5] |
| Has Ppl Per Token | 0.64 | [6] |
| Has Same Model Quality As | Lohe Spherical | [6] |
| Converges Identically to | Lohe Spherical | [6] |
| Has Param Count Approx | 17M | [6] |
| Has Ppl | 8228 | [6] |
| Has Best Quality | true | [7] |
| Has Ppl | 6.03 | [7] |
| Has Tokens Per Second at Temp256 | 210000 | [7] |
| Has Quality Adjusted Tokens Per Second | 8008 | [7] |
| Has Tok Per S | 56931 | [8] |
| Has Final Ppl | 239 | [8] |
| Has Params | 16.9M | [8] |
| Second Best in Ppl | null | [8] |
| Uses | Funk Hecke Decomposition | [8] |
| Has Sync Hub Location | Block 10 | [10] |
| Has Sync Strength | 0.56 | [10] |
| Has Baseline Throughput | 12000 | [10] |
| Has Throughput Unit | Tok S | [10] |
| Achieved Perplexity | 116.4 | [11] |
| Has Speedup Factor | 120 | [11] |
| Benefited From | Parallel Prefill | [11] |
| Compatible With | Rope Py | [11] |
| Uses Rope Mode | True Rope Qk | [11] |
| Validation Perplexity | 116.4 | [11] |
| Converges Smoothly | true | [11] |
| Has Final Order | 0.36 | [11] |
| Phase State | disordered | [11] |
| Classified As | Better Language Model | [11] |
| Has Perplexity | 8228 | [12] |
| Has Ppl Per Token | 0.64 | [12] |
| Outputs Pattern | Word Like Runs | [13] |
| Outputs Sample Text | Spectral Output Sample | [13] |
| Tends Toward Output Pattern | Run on Noun Phrases | [13] |
| Described Speed | Faster | [13] |
| Approximate Throughput | 400 | [13] |
| Output Length | longer | [13] |
| Appears in | Code Block | [13] |
| Outputs Sample | Spectral Output Sample | [13] |
| Precedes | Anchor Kan | [13] |
| Output Characteristic | longer word-like runs | [13] |
| Example Output | The not used in the be 2001 3 -5 of a same | [13] |
| Includes | numbers and partial structure | [13] |
| Tends Toward | run-on noun phrases | [13] |
| Performance | faster | [13] |
Timeline
Timeline axis is valid_time — when each source says the fact was true in the world, not when Dontopedia learned about it. Retracted rows are kept for provenance; coloured stripes indicate the context kind.
References (13)
ctx:discord/blah/watt-activation/part-104ctx:discord/blah/watt-activation/part-125ctx:discord/blah/watt-activation/part-221ctx:discord/blah/watt-activation/part-232ctx:discord/blah/watt-activation/part-236ctx:discord/blah/watt-activation/part-317ctx:discord/blah/watt-activation/part-103ctx:discord/blah/watt-activation/part-319ctx:discord/blah/watt-activation/204- full textwatt-activation-204text/plain3 KB
doc:agent/watt-activation-204/c8f804b0-ce31-4b4d-9ff6-960a682fa16dShow excerpt
[2026-03-10 07:26] xenonfun: those are all with `opt = RotationalAdamW(model, lr=1e-4, weight_decay=0.01, zero_mean=True)` optimizer. ``` ⏺ Right — the whole point of Form 2 was to give LoheOptimizer a real order parameter instead of the …
ctx:discord/blah/watt-activation/220- full textwatt-activation-220text/plain3 KB
doc:agent/watt-activation-220/5c7f4a28-90e7-46de-ae1e-9e19a58c8d65Show excerpt
[2026-03-11 04:42] xenonfun: FFN DFT — much richer specialization than spectral: ``` ┌─────┬───────┬────────┬────────────────┬────────────────┐ │ blk │ r │ FFN DC │ dominant mode │ pattern │ ├─────┼───────┼────────┼───────…
ctx:discord/blah/watt-activation/235- full textwatt-activation-235text/plain2 KB
doc:agent/watt-activation-235/2b477fba-6b21-4139-a3e7-6c48c5dad99fShow excerpt
[2026-03-11 22:42] xenonfun: ⏺ Done. Pushed 34b7246. Summary of what landed: - Results: anchor_kan best ppl 155.6 / 52K tok/s, spectral 116.4 / 43.7K tok/s - parallel prefill: 120× faster for spectral, auto-fallback for anchor_kan - …
ctx:discord/blah/watt-activation/315- full textwatt-activation-315text/plain2 KB
doc:agent/watt-activation-315/5f801374-2b73-4ce9-bd06-bdb14c70d743Show excerpt
[2026-03-15 01:50] xenonfun: ``` ⏺ Attention Type Comparison (200 steps, d=768, 12L, 12H, compiled) ┌────────────────┬──────┬───────┬────────┬────────┬─────────┐ │ Attention │ Loss │ PPL │ Tok/s │ Params │ PPL/tok │ ├────────…
ctx:claims/document/019a8f6a-d43b-40ba-afc1-247f4b73c3a5- full textxenonfun: well not that much speed up 46K now peak, think its memory bound already. 8K voctext/plain680 B
discord:msg/a3126764-fdd1-42f8-9653-a5170ea5bdefShow excerpt
xenonfun: well not that much speed up 46K now peak, think its memory bound already. 8K vocab is signifigantly worse output at same training but makes sense, model was too lopsided with that much embeddings, but at 100K it did observe intere…
See also
- Pytorch Mps
- Super Linear Ish
- Memory Standard
- Gpt
- Anchor Kan
- Block 10
- Anchorkan
- L2
- L2 H0
- Lohe Spherical
- Funk Hecke Decomposition
- Model Component
- Attention Configuration
- Tok S
- Model Architecture
- Parallel Prefill
- Rope Py
- True Rope Qk
- Better Language Model
- Attention Type
- Model Kind
- Word Like Runs
- Spectral Output Sample
- Run on Noun Phrases
- Code Block
- Model
Keep researching
Missing something or suspicious of what's here? Kick off a research session — a Claude agent will investigate, cite its sources, and file new facts into a dedicated context you can review before accepting into the shared view.