Dontopedia

HarmonicGPT

From Dontopedia, the open, paraconsistent wiki. (Last updated 2026-06-06.)

Linked via sameAs to 1 other subject: Spectral ReservoirReview & merge →

HarmonicGPT is configurable attention GPT with weight-tied embeddings.

59 facts·49 predicates·18 sources·4 in dispute

Mostly:rdf:type(4), has recent pull request(3), uses attention(2)

Maturity scale raw canonical shape-checked rule-derived certified

Inbound mentions (35)

Other subjects in dontopedia point AT this entity as a value. These are inverse relationships — e.g. "X motherOf this subject" — and answer questions the forward facts can't. Grouped by predicate.

trainsModelTrains Model(6)

isAttentionTypeIs Attention Type(2)

akaAka(1)

announcesPushingPrAnnounces Pushing Pr(1)

appliesToApplies to(1)

basedOnBased on(1)

compatibleWithHarmonicGPTCompatible With Harmonic Gpt(1)

contributesToContributes to(1)

developedInContextOfDeveloped in Context of(1)

divergedFromDiverged From(1)

existInRepoExist in Repo(1)

feelsPrettyHappyFeels Pretty Happy(1)

hasAliasHas Alias(1)

hasModelHas Model(1)

hasSameIssueHas Same Issue(1)

hasTopicHas Topic(1)

includesPerLayerCouplingIncludes Per Layer Coupling(1)

isFromIs From(1)

isKnownToExistInReferenceIs Known to Exist in Reference(1)

isSharedByIs Shared by(1)

originatesFromReferenceImplementationOriginates From Reference Implementation(1)

ownsRepoOwns Repo(1)

presupposesReferenceImplementationPresupposes Reference Implementation(1)

referencesReferences(1)

referencesExternalWorkReferences External Work(1)

sharesMathWithShares Math With(1)

sharesTrainingResultsShares Training Results(1)

sourceOfSource of(1)

testsForwardPassTests Forward Pass(1)

Other facts (57)

The long tail: predicates that appear too rarely to warrant their own section. Filter or scroll to find a specific one. Each row links to its source.

57 facts
PredicateValueRef
Rdf:typeSoftware System[15]
Rdf:typeModel[16]
Rdf:typeSoftware Project[17]
Rdf:typeSoftware Project[18]
Has Recent Pull RequestPr 7[18]
Has Recent Pull RequestPr 8[18]
Has Recent Pull RequestPr 9[18]
Uses Attentionlinear[7]
Uses AttentionLohe Spherical[8]
Has Model Count33[13]
Has Model Count33[18]
Has Optimizer Count9[13]
Has Optimizer Count9[18]
Uses Fft MachineryFft[1]
Is RepoMonumental Systems[2]
Uses Weight Tied EmbeddingsEmbeddings[3]
Is Configurable Attention GptAttention Gpt[3]
Has Dimension D768[4]
Has Num Heads H12[4]
Is Transformer Varianttrue[4]
Has Num Layers L12[4]
Has Default Max Seq Len512[5]
Requires Always Pass Max Seq Len Ge Seq LenUsage Rule[5]
Uses Attention MechanismResonance Attention[6]
References Known Modelnull[6]
Has Dimension768[7]
Has Heads12[7]
Is Neural Networktransformer-like[7]
Has Sequence Length2048[7]
References ProjectHarmonicGPT[7]
Has Layers12[7]
Is Model Varianttrue[9]
Will IntegrateSpherical Vq Bottleneck[10]
Requires IntegrationVq Bottleneck Integration[10]
ProvidesGivens Unitary Class[11]
References Prior Work{}[11]
Has Reference Implementation42 features[12]
Has Paper Draft Count8[13]
Has Access Levelprivate[13]
Has Diverged FromHarmonic Rust[13]
Has Layer Count Approx50+[13]
Contrasts WithHarmonic Rust[13]
Is Written inPython/PyTorch[13]
Serves Roleresearch lab[13]
Shares Mathematics WithHarmonic Rust[13]
Has Working BestGrouped1[14]
Solves ProblemPyTorch/CUDA problem[15]
Descriptionconfigurable attention GPT with weight-tied embeddings[16]
Has ComponentHarmonic Gpt Mlx Backend[17]
Has Implementation LanguagePython[18]
Uses FrameworkPy Torch[18]
Has Access StatusPrivate[18]
Has RoleResearch Lab[18]
Has Layer Count50[18]
Has Draft Paper Count8[18]
Shares Math WithHarmonic Rust[18]
Diverged FromHarmonic Rust[18]

Timeline

Timeline axis is valid_time — when each source says the fact was true in the world, not when Dontopedia learned about it. Retracted rows are kept for provenance; coloured stripes indicate the context kind.

59 undated
—
usesFftMachineryblah/watt-activation/part-101
ex:fft
—
isRepoblah/watt-activation/part-104
ex:monumental-systems
—
usesWeightTiedEmbeddingsblah/watt-activation/part-105
ex:embeddings
—
isConfigurableAttentionGPTblah/watt-activation/part-105
ex:attention-gpt
—
hasDimensionDblah/watt-activation/part-112
768
—
hasNumHeadsHblah/watt-activation/part-112
12
—
isTransformerVariantblah/watt-activation/part-112
true
—
hasNumLayersLblah/watt-activation/part-112
12
—
hasDefaultMaxSeqLenblah/watt-activation/part-119
512
—
requiresAlwaysPassMaxSeqLenGeSeqLenblah/watt-activation/part-119
ex:usage-rule
—
usesAttentionMechanismblah/watt-activation/part-111
ex:resonance-attention
—
referencesKnownModelblah/watt-activation/part-111
null
—
hasDimensionblah/watt-activation/part-114
768
—
hasHeadsblah/watt-activation/part-114
12
—
isNeuralNetworkblah/watt-activation/part-114
transformer-like
—
hasSequenceLengthblah/watt-activation/part-114
2048
—
referencesProjectblah/watt-activation/part-114
HarmonicGPT
—
usesAttentionblah/watt-activation/part-114
linear
—
hasLayersblah/watt-activation/part-114
12
—
usesAttentionblah/watt-activation/part-201
ex:lohe-spherical
—
isModelVariantblah/watt-activation/part-223
true
—
willIntegrateblah/watt-activation/part-279
ex:spherical-vq-bottleneck
—
requiresIntegrationblah/watt-activation/part-279
ex:vq-bottleneck-integration
—
providesblah/watt-activation/part-384
ex:givens-unitary-class
—
referencesPriorWorkblah/watt-activation/part-384
{}
—
hasReferenceImplementationblah/watt-activation/part-385
42 features
—
hasPaperDraftCountblah/watt-activation/part-492
8
—
hasAccessLevelblah/watt-activation/part-492
private
—
hasDivergedFromblah/watt-activation/part-492
ex:harmonic-rust
—
hasLayerCountApproxblah/watt-activation/part-492
50+
—
hasModelCountblah/watt-activation/part-492
33
—
hasOptimizerCountblah/watt-activation/part-492
9
—
contrastsWithblah/watt-activation/part-492
ex:harmonic-rust
—
isWrittenInblah/watt-activation/part-492
Python/PyTorch
—
servesRoleblah/watt-activation/part-492
research lab
—
sharesMathematicsWithblah/watt-activation/part-492
ex:harmonic-rust
—
hasWorkingBestblah/watt-activation/part-103
ex:grouped1
—
labelblah/watt-activation/101
harmonic-gpt
—
typeblah/watt-activation/101
ex:SoftwareSystem
—
solvesProblemblah/watt-activation/101
PyTorch/CUDA problem
—
typeblah/watt-activation/105
ex:Model
—
labelblah/watt-activation/105
HarmonicGPT
—
descriptionblah/watt-activation/105
configurable attention GPT with weight-tied embeddings
—
typeblah/watt-activation/106
ex:SoftwareProject
—
hasComponentblah/watt-activation/106
ex:harmonic-gpt-mlx-backend
—
typeblah/watt-activation/489
ex:SoftwareProject
—
hasImplementationLanguageblah/watt-activation/489
ex:Python
—
usesFrameworkblah/watt-activation/489
ex:PyTorch
—
hasAccessStatusblah/watt-activation/489
ex:private
—
hasRoleblah/watt-activation/489
ex:research-lab
—
hasModelCountblah/watt-activation/489
33
—
hasLayerCountblah/watt-activation/489
50
—
hasOptimizerCountblah/watt-activation/489
9
—
hasDraftPaperCountblah/watt-activation/489
8
—
sharesMathWithblah/watt-activation/489
ex:HarmonicRust
—
divergedFromblah/watt-activation/489
ex:HarmonicRust
—
hasRecentPullRequestblah/watt-activation/489
ex:PR-7
—
hasRecentPullRequestblah/watt-activation/489
ex:PR-8
—
hasRecentPullRequestblah/watt-activation/489
ex:PR-9

References (18)

18 references
  1. [1]Part 1011 fact
    ctx:discord/blah/watt-activation/part-101
  2. [2]Part 1041 fact
    ctx:discord/blah/watt-activation/part-104
  3. [3]Part 1052 facts
    ctx:discord/blah/watt-activation/part-105
  4. [4]Part 1124 facts
    ctx:discord/blah/watt-activation/part-112
  5. [5]Part 1192 facts
    ctx:discord/blah/watt-activation/part-119
  6. [6]Part 1112 facts
    ctx:discord/blah/watt-activation/part-111
  7. [7]Part 1147 facts
    ctx:discord/blah/watt-activation/part-114
  8. [8]Part 2011 fact
    ctx:discord/blah/watt-activation/part-201
  9. [9]Part 2231 fact
    ctx:discord/blah/watt-activation/part-223
  10. [10]Part 2792 facts
    ctx:discord/blah/watt-activation/part-279
  11. [11]Part 3842 facts
    ctx:discord/blah/watt-activation/part-384
  12. [12]Part 3851 fact
    ctx:discord/blah/watt-activation/part-385
  13. [13]Part 49210 facts
    ctx:discord/blah/watt-activation/part-492
  14. [14]Part 1031 fact
    ctx:discord/blah/watt-activation/part-103
  15. [15]1013 facts
    ctx:discord/blah/watt-activation/101
    • full textwatt-activation-101
      text/plain3 KBdoc:agent/watt-activation-101/6f90e9c2-2d77-484f-96fd-98a89c99440a
      Show excerpt
      [2026-03-08 18:08] xenonfun: ### Concrete comparison at T=4096, H=12, d_h=64: ResonanceV2 (cumsum, K=8 bands): Work: 12 × 8 × 4096 × 64 = 25M multiply-adds Kernel launches: ~K cumsums = 8 Metal kernels Memory: O(T · H · K · …
  16. [16]1053 facts
    ctx:discord/blah/watt-activation/105
    • full textwatt-activation-105
      text/plain3 KBdoc:agent/watt-activation-105/561920dc-7f65-4ab4-80fa-8e3162aa9046
      Show excerpt
      [2026-03-08 19:26] xenonfun: ``` What They're Leaving on the Table 1. No mx.compile — Their benchmark and model run eagerly. From our experience with AnchorKAN at similar scale, compiled step gives ~1.5-2x throughput improvement on M…
  17. [17]1062 facts
    ctx:discord/blah/watt-activation/106
    • full textwatt-activation-106
      text/plain3 KBdoc:agent/watt-activation-106/1cc0dbfa-458e-4f02-b7b6-1c37e3d3a7f8
      Show excerpt
      [2026-03-08 19:52] xenonfun: ``` total used free wired compressed 98304Mi 88429Mi 8982Mi 59988Mi 100Mi ``` (files: Screenshot_2026-03-08_at_3.52.13_PM.png) [2026-03-08 19:57] xenon…
  18. [18]48914 facts
    ctx:discord/blah/watt-activation/489
    • full textwatt-activation-489
      text/plain3 KBdoc:agent/watt-activation-489/157c31fa-da74-456d-872d-78bc37e44f45
      Show excerpt
      [2026-03-22 05:24] xenonfun: still got some gaps after that first pass (files: Screenshot_2026-03-22_at_1.23.51_AM.png) [2026-03-22 05:24] xenonfun: we got fucking crazy amount of log detail [2026-03-22 05:26] lisamegawatts: this is getting…

See also

Keep researching

Missing something or suspicious of what's here? Kick off a research session — a Claude agent will investigate, cite its sources, and file new facts into a dedicated context you can review before accepting into the shared view.

↑