Dontopedia

LinearAttention

From Dontopedia, the open, paraconsistent wiki. (Last updated 2026-06-06.)

LinearAttention is full d_h x d_h outer product.

15 facts·14 predicates·5 sources

Mostly:uses full outer product(1), enables long context(1), has key property(1)

Maturity scale raw canonical shape-checked rule-derived certified

Inbound mentions (5)

Other subjects in dontopedia point AT this entity as a value. These are inverse relationships — e.g. "X motherOf this subject" — and answer questions the forward facts can't. Grouped by predicate.

complexityClassComplexity Class(1)

hasAttentionTypeHas Attention Type(1)

implementsAttentionTypeImplements Attention Type(1)

includesLinearAttentionIncludes Linear Attention(1)

usesMechanismUses Mechanism(1)

Other facts (14)

The long tail: predicates that appear too rarely to warrant their own section. Filter or scroll to find a specific one. Each row links to its source.

14 facts
PredicateValueRef
Uses Full Outer Productd_h x d_h[1]
Enables Long ContextMillion Tokens[2]
Has Key PropertyO1 Memory Generate Step[2]
Lacks Kv Cache ProblemStandard Transformers[2]
Weakens SignalText Signal[3]
Rdf:typeAttention Variant[4]
Descriptionfull d_h x d_h outer product[4]
Key PropertyConstant Memory Inference[5]
Inference Memory ComplexityOrder 1[5]
Memory Complexity ContextGenerate Step Mode[5]
Supports Long Context1000000[5]
Memory Constant Over Contexttrue[5]
Cost of Longer ContextLarger Positional Embedding Table[5]
Lacks ProblemKv Cache Growth[5]

Timeline

Timeline axis is valid_time — when each source says the fact was true in the world, not when Dontopedia learned about it. Retracted rows are kept for provenance; coloured stripes indicate the context kind.

15 undated
—
usesFullOuterProductblah/watt-activation/part-105
d_h x d_h
—
enablesLongContextblah/watt-activation/part-126
ex:million-tokens
—
hasKeyPropertyblah/watt-activation/part-126
ex:o1-memory-generate-step
—
lacksKvCacheProblemblah/watt-activation/part-126
ex:standard-transformers
—
weakensSignalblah/watt-activation/part-255
ex:text-signal
—
typeblah/watt-activation/105
ex:AttentionVariant
—
labelblah/watt-activation/105
LinearAttention
—
descriptionblah/watt-activation/105
full d_h x d_h outer product
—
keyPropertyblah/watt-activation/126
ex:constant-memory-inference
—
inferenceMemoryComplexityblah/watt-activation/126
ex:order-1
—
memoryComplexityContextblah/watt-activation/126
ex:generate-step-mode
—
supportsLongContextblah/watt-activation/126
1000000
—
memoryConstantOverContextblah/watt-activation/126
true
—
costOfLongerContextblah/watt-activation/126
ex:larger-positional-embedding-table
—
lacksProblemblah/watt-activation/126
ex:kv-cache-growth

References (5)

5 references
  1. [1]Part 1051 fact
    ctx:discord/blah/watt-activation/part-105
  2. [2]Part 1263 facts
    ctx:discord/blah/watt-activation/part-126
  3. [3]Part 2551 fact
    ctx:discord/blah/watt-activation/part-255
  4. [4]1053 facts
    ctx:discord/blah/watt-activation/105
    • full textwatt-activation-105
      text/plain3 KBdoc:agent/watt-activation-105/561920dc-7f65-4ab4-80fa-8e3162aa9046
      Show excerpt
      [2026-03-08 19:26] xenonfun: ``` What They're Leaving on the Table 1. No mx.compile — Their benchmark and model run eagerly. From our experience with AnchorKAN at similar scale, compiled step gives ~1.5-2x throughput improvement on M…
  5. [5]1267 facts
    ctx:discord/blah/watt-activation/126
    • full textwatt-activation-126
      text/plain3 KBdoc:agent/watt-activation-126/dddfc295-807c-4943-b01a-f4f0a977c17e
      Show excerpt
      [2026-03-09 04:03] xenonfun: ### What context count we do at this scale? ⏺ From the measurements we have, memory scales roughly linearly with total tokens in the batch: - BS=4, seq=1024 → 4,096 tokens → ~40 GB - BS=8, seq=1024 → 8,192 …

See also

Keep researching

Missing something or suspicious of what's here? Kick off a research session — a Claude agent will investigate, cite its sources, and file new facts into a dedicated context you can review before accepting into the shared view.

↑