Dontopedia

KANAttention

From Dontopedia, the open, paraconsistent wiki. (Last updated 2026-06-06.)

KANAttention is Chebyshev polynomial feature maps.

34 facts·26 predicates·9 sources·2 in dispute

Mostly:rdf:type(4), has near zero cost(1), allows network to learn(1)

Maturity scale raw canonical shape-checked rule-derived certified

Inbound mentions (15)

Other subjects in dontopedia point AT this entity as a value. These are inverse relationships — e.g. "X motherOf this subject" — and answer questions the forward facts can't. Grouped by predicate.

hasAttentionTypeHas Attention Type(1)

implementsAttentionTypeImplements Attention Type(1)

includesKANAttentionIncludes Kan Attention(1)

involvesAlgorithmInvolves Algorithm(1)

isInIs in(1)

isIntellectualPathOnlyIs Intellectual Path Only(1)

isPartOfLargerArchitectureIs Part of Larger Architecture(1)

learnsAttentionSharpeningFunctionLearns Attention Sharpening Function(1)

ledIntellectuallyToLed Intellectually to(1)

ledToLed to(1)

possessesLearnableAttentionKernelPossesses Learnable Attention Kernel(1)

presupposesKnowledgeOfPresupposes Knowledge of(1)

referencesCodeInReferences Code in(1)

statesActionableTakeawayStates Actionable Takeaway(1)

unnecessaryForEndProductUnnecessary for End Product(1)

Other facts (29)

The long tail: predicates that appear too rarely to warrant their own section. Filter or scroll to find a specific one. Each row links to its source.

29 facts
PredicateValueRef
Rdf:typeTechnique[5]
Rdf:typeAttention Mechanism[6]
Rdf:typeCode Module[7]
Rdf:typeAlgorithm[9]
Has Near Zero CostParameters[1]
Allows Network to Learnown attention sharpening function[1]
Is Simpler ThanOscillator Dynamics[1]
Is Near Zero Cost Drop in ImprovementStandard Attention[1]
Is Actionable TakeawayLearnable Polynomial Attention Kernels[1]
Uses Python LoopChebyshev Features[2]
Uses Chebyshev Polynomial Feature MapsChebyshev Polynomials[2]
Changes Chebyshev InputTanh Q[3]
Is100x Cheaper ThanCurrent O T2 Loheattention[4]
Has Flat Scaling in Ttrue[4]
Commits to Flat Compute ScalingSequence Length T[4]
Dominant Cost IsO T D2 Linear Projections[4]
Is Superior in EfficiencyLoheattention[4]
Has Descriptionlearnable polynomial attention kernels[5]
ImprovesStandard Attention[5]
Compares toOscillator Dynamics[5]
Perplexity StatusOOM[6]
Tokens Per Second82000[6]
Quality Adjusted Tokens Per Second StatusOOMs at scale[6]
Has MethodChebyshev Features Method[7]
DescriptionChebyshev polynomial feature maps[7]
Cost RelativeLoheattention[9]
Relative Cost Multiplier0.01[9]
Scaling Behavioressentially flat in T[9]
Dominant Cost ComponentO(T·d²) Linear projections[9]

Timeline

Timeline axis is valid_time — when each source says the fact was true in the world, not when Dontopedia learned about it. Retracted rows are kept for provenance; coloured stripes indicate the context kind.

34 undated
—
hasNearZeroCostblah/watt-activation/part-63
ex:parameters
—
allowsNetworkToLearnblah/watt-activation/part-63
own attention sharpening function
—
isSimplerThanblah/watt-activation/part-63
ex:oscillator-dynamics
—
isNearZeroCostDropInImprovementblah/watt-activation/part-63
ex:standard-attention
—
isActionableTakeawayblah/watt-activation/part-63
ex:learnable-polynomial-attention-kernels
—
usesPythonLoopblah/watt-activation/part-105
ex:_chebyshev-features
—
usesChebyshevPolynomialFeatureMapsblah/watt-activation/part-105
ex:chebyshev-polynomials
—
changesChebyshevInputblah/watt-activation/part-118
ex:tanh-q
—
is100xCheaperThanblah/watt-activation/part-646
ex:current-o-t2-loheattention
—
hasFlatScalingInTblah/watt-activation/part-646
true
—
commitsToFlatComputeScalingblah/watt-activation/part-646
ex:sequence-length-t
—
dominantCostIsblah/watt-activation/part-646
ex:o-t-d2-linear-projections
—
isSuperiorInEfficiencyblah/watt-activation/part-646
ex:loheattention
—
typeblah/watt-activation/63
ex:Technique
—
labelblah/watt-activation/63
KAN-Attention
—
hasDescriptionblah/watt-activation/63
learnable polynomial attention kernels
—
improvesblah/watt-activation/63
ex:standard-attention
—
comparesToblah/watt-activation/63
ex:oscillator-dynamics
—
labelblah/watt-activation/103
KAN (quadratic)
—
typeblah/watt-activation/103
ex:AttentionMechanism
—
perplexityStatusblah/watt-activation/103
OOM
—
tokensPerSecondblah/watt-activation/103
82000
—
qualityAdjustedTokensPerSecondStatusblah/watt-activation/103
OOMs at scale
—
typeblah/watt-activation/105
ex:CodeModule
—
hasMethodblah/watt-activation/105
ex:chebyshev-features-method
—
labelblah/watt-activation/105
KANAttention
—
descriptionblah/watt-activation/105
Chebyshev polynomial feature maps
—
labelblah/watt-activation/118
KANAttention
—
typeblah/watt-activation/643
ex:Algorithm
—
labelblah/watt-activation/643
KAN attention
—
costRelativeblah/watt-activation/643
ex:loheattention
—
relativeCostMultiplierblah/watt-activation/643
0.01
—
scalingBehaviorblah/watt-activation/643
essentially flat in T
—
dominantCostComponentblah/watt-activation/643
O(T·d²) Linear projections

References (9)

9 references
  1. [1]Part 635 facts
    ctx:discord/blah/watt-activation/part-63
  2. [2]Part 1052 facts
    ctx:discord/blah/watt-activation/part-105
  3. [3]Part 1181 fact
    ctx:discord/blah/watt-activation/part-118
  4. [4]Part 6465 facts
    ctx:discord/blah/watt-activation/part-646
  5. [5]635 facts
    ctx:discord/blah/watt-activation/63
    • full textwatt-activation-63
      text/plain2 KBdoc:agent/watt-activation-63/1bd53136-248b-4353-b53c-b8c81b9d26f4
      Show excerpt
      [2026-03-07 15:30] xenonfun: ### no GPU contention run ``` Clean results at 5K iters: ┌───────────────────┬──────┬───────┬──────────┬───────────┬──────────┬───────┬─────────┐ │ Config │ it/s │ tok/s │ Avg Loss │ Final PPL …
  6. [6]1035 facts
    ctx:discord/blah/watt-activation/103
    • full textwatt-activation-103
      text/plain3 KBdoc:agent/watt-activation-103/6d322edd-8b82-4859-be6f-bc7033a53fe1
      Show excerpt
      [2026-03-08 18:36] xenonfun: It appears your agents have actually already done all this work <@1211062099137265723> in your repo already. https://github.com/MonumentalSystems/harmonic-gpt/blob/master/docs/M3_DEPLOY_NOTES.md (files: Screens…
  7. [7]1054 facts
    ctx:discord/blah/watt-activation/105
    • full textwatt-activation-105
      text/plain3 KBdoc:agent/watt-activation-105/561920dc-7f65-4ab4-80fa-8e3162aa9046
      Show excerpt
      [2026-03-08 19:26] xenonfun: ``` What They're Leaving on the Table 1. No mx.compile — Their benchmark and model run eagerly. From our experience with AnchorKAN at similar scale, compiled step gives ~1.5-2x throughput improvement on M…
  8. [8]1181 fact
    ctx:discord/blah/watt-activation/118
    • full textwatt-activation-118
      text/plain3 KBdoc:agent/watt-activation-118/ed79098d-1144-44f5-9941-e6b2b9c1caa7
      Show excerpt
      [2026-03-08 23:43] xenonfun: Code Changes (3 important patterns) 1. Fused QKV projection in SpectralAttention - Separate q_proj, k_proj, v_proj → single qkv_proj = Linear(d_model, 3 * d_model). One matmul instead of three. We should po…
  9. [9]6436 facts
    ctx:discord/blah/watt-activation/643

See also

Keep researching

Missing something or suspicious of what's here? Kick off a research session — a Claude agent will investigate, cite its sources, and file new facts into a dedicated context you can review before accepting into the shared view.

↑