Code LM

as donto believed on

From Dontopedia, the open, paraconsistent wiki. (Last updated 2026-06-06.)

Code LM has 39 facts recorded in Dontopedia across 13 references.

39 facts·35 predicates·13 sources

Mostly:generates(2), improves with context(2), has output head currently(1)

Inbound mentions (12)

Other subjects in dontopedia point AT this entity as a value. These are inverse relationships — e.g. "X motherOf this subject" — and answer questions the forward facts can't. Grouped by predicate.

`branchesTo`Branches to(1)

Code Index K Starex:code-index-k-star

`compares`Compares(1)

Context Scaling Tableex:context-scaling-table

`decreasesWithContextSize`Decreases With Context Size(1)

Code Lm Pplex:code-lm-ppl

`hasComponent`Has Component(1)

Generation Pipeline Diagramex:generation-pipeline-diagram

`inputTo`Input to(1)

Structural Codesex:structural-codes

`is`Is(1)

Pipeline Stage 4ex:pipeline-stage-4

`outputsTo`Outputs to(1)

Spherical Vqex:spherical-vq

`precedesStage`Precedes Stage(1)

Spherical Vqex:spherical-vq

`presupposesExistenceOf`Presupposes Existence of(1)

Textex:text

`sendsDataTo`Sends Data to(1)

Spherical Vq Bottleneckex:spherical-vq-bottleneck

`stage5`Stage5(1)

Processing Pipelineex:processing-pipeline

`targetsModels`Targets Models(1)

Rjs Longgenex:rjs-longgen

Other facts (37)

The long tail: predicates that appear too rarely to warrant their own section. Filter or scroll to find a specific one. Each row links to its source.

37 facts

Predicate	Value	Ref
Generates	Code Grammar	[4]
Generates	Code Grammar	[11]
Improves With Context	true	[5]
Improves With Context	true	[10]
Has Output Head Currently	Tied Embedding	[1]
Change Suggested	null	[1]
Benefits From Context Expansion	16x	[2]
Has Pp Limprovement	1.15x	[2]
Improves More Than Byte Lm	null	[2]
Scales With Context	null	[2]
Depends on	Vq Encoder	[3]
Generates Code Grammar	null	[4]
Output to	Ar Byte Decoder	[4]
Superior to	Byte Lm	[5]
Has Low Perplexity	true	[5]
Has Nearly Deterministic Grammar	true	[5]
Reaches Ppl	1.71	[5]
Is Autoregressive	Next Code Prediction	[6]
Uses	Lohespherical	[6]
Has	Vmf Head	[6]
Predicts	Next Code	[6]
Matches Geometry	S D 1 Geometry	[7]
Now Uses	Cos H W Tau Head	[7]
Uses Output Head	cos(h, w) / tau	[7]
Has Ppl	2.1	[7]
Is Healthy	Ppl Metric	[7]
Precedes Stage	Lohe Ar Decoder	[8]
Has Head	Vmf Head	[8]
Rdf:type	Language Model	[9]
Has Component	Output Head	[9]
Has Perplexity	1.71	[10]
Has Head Type	Vmf Head	[12]
Outputs to	Lohe Ar Decoder	[12]
Has Task	predict next code	[13]
Has Architecture	LoheSpherical	[13]
Has Output Head	vMF head	[13]
Performs Operation	sample code sequence	[13]

Timeline axis is valid_time — when each source says the fact was true in the world, not when Dontopedia learned about it. Retracted rows are kept for provenance; coloured stripes indicate the context kind.

39 undated

—

hasOutputHeadCurrentlyblah/watt-activation/part-291

ex:tied-embedding

—

changeSuggestedblah/watt-activation/part-291

null

—

benefitsFromContextExpansionblah/watt-activation/part-298

16x

—

hasPPLimprovementblah/watt-activation/part-298

1.15x

—

improvesMoreThanByteLMblah/watt-activation/part-298

null

—

scalesWithContextblah/watt-activation/part-298

null

—

dependsOnblah/watt-activation/part-299

ex:vq-encoder

—

generatesCodeGrammarblah/watt-activation/part-300

null

—

generatesblah/watt-activation/part-300

ex:code-grammar

—

outputToblah/watt-activation/part-300

ex:ar-byte-decoder

—

superiorToblah/watt-activation/part-296

ex:byte-lm

—

hasLowPerplexityblah/watt-activation/part-296

true

—

hasNearlyDeterministicGrammarblah/watt-activation/part-296

true

—

improvesWithContextblah/watt-activation/part-296

true

—

reachesPplblah/watt-activation/part-296

1.71

—

isAutoregressiveblah/watt-activation/part-305

ex:next-code-prediction

—

usesblah/watt-activation/part-305

ex:lohespherical

—

hasblah/watt-activation/part-305

ex:vmf-head

—

predictsblah/watt-activation/part-305

ex:next-code

—

matchesGeometryblah/watt-activation/part-292

ex:s-d-1-geometry

—

nowUsesblah/watt-activation/part-292

ex:cos-h-w-tau-head

—

usesOutputHeadblah/watt-activation/part-292

cos(h, w) / tau

—

hasPplblah/watt-activation/part-292

2.1

—

isHealthyblah/watt-activation/part-292

ex:ppl-metric

—

precedesStageblah/watt-activation/part-302

ex:lohe-ar-decoder

—

hasHeadblah/watt-activation/part-302

ex:vmf-head

—

typeblah/watt-activation/289

ex:LanguageModel

—

labelblah/watt-activation/289

Code LM

—

hasComponentblah/watt-activation/289

ex:output-head

—

improvesWithContextblah/watt-activation/294

true

—

hasPerplexityblah/watt-activation/294

1.71

—

labelblah/watt-activation/298

Code LM

—

generatesblah/watt-activation/298

ex:code-grammar

—

hasHeadTypeblah/watt-activation/300

ex:vmf-head

—

outputsToblah/watt-activation/300

ex:lohe-ar-decoder

—

hasTaskblah/watt-activation/303

predict next code

—

hasArchitectureblah/watt-activation/303

LoheSpherical

—

hasOutputHeadblah/watt-activation/303

vMF head

—

performsOperationblah/watt-activation/303

sample code sequence

References (13)

13 references

[1]Part 2912 facts
ctx:discord/blah/watt-activation/part-291
[2]Part 2984 facts
ctx:discord/blah/watt-activation/part-298
[3]Part 2991 fact
ctx:discord/blah/watt-activation/part-299
[4]Part 3003 facts
ctx:discord/blah/watt-activation/part-300
[5]Part 2965 facts
ctx:discord/blah/watt-activation/part-296
[6]Part 3054 facts
ctx:discord/blah/watt-activation/part-305
[7]Part 2925 facts
ctx:discord/blah/watt-activation/part-292
[8]Part 3022 facts
ctx:discord/blah/watt-activation/part-302
[9]2893 facts
ctx:discord/blah/watt-activation/289
- full textwatt-activation-289
  text/plain3 KBdoc:agent/watt-activation-289/d4d5a987-9de3-4ba0-bff6-2dedbb3e5506
  Show excerpt
  [2026-03-14 02:51] xenonfun: ``` what is our Cookbook r_global ? and why is it ⏺ Codebook r_global is the Kuramoto order parameter of the codebook entries themselves — the same r_global used throughout the Lohe dynamics in LoheSpherical…
[10]2942 facts
ctx:discord/blah/watt-activation/294
- full textwatt-activation-294
  text/plain3 KBdoc:agent/watt-activation-294/69ba78b6-677d-48bc-bf97-4a2f034032b1
  Show excerpt
  [2026-03-14 04:25] xenonfun: ``` Context Scaling ┌─────────────────┬─────────────┬──────┐ │ Code LM Context │ Equiv Bytes │ PPL │ ├─────────────────┼─────────────┼──────┤ │ 32 │ 256 │ 1.96 │ ├───────────────…
[11]2982 facts
ctx:discord/blah/watt-activation/298
- full textwatt-activation-298
  text/plain2 KBdoc:agent/watt-activation-298/f5cde311-fd9a-43e7-a746-9177b5a91fee
  Show excerpt
  [2026-03-14 05:53] xenonfun: ``` What Changed The AR decoder produces recognizable English word fragments: "the", "and", "for", "with", "this", "one", "protec(tion)", "earch", "context", "project", "state", "imported". These are real m…
[12]3002 facts
ctx:discord/blah/watt-activation/300
- full textwatt-activation-300
  text/plain3 KBdoc:agent/watt-activation-300/3b6edccf-3524-4608-838f-25890efaea15
  Show excerpt
  [2026-03-14 06:34] xenonfun: ``` 3. Manual attention (lines 110-128) — Hand-rolled softmax attention instead of using mx.fast.scaled_dot_product_attention. MLX's fused attention kernel is significantly faster for small sequence lengths. …
[13]3034 facts
ctx:discord/blah/watt-activation/303
- full textwatt-activation-303
  text/plain3 KBdoc:agent/watt-activation-303/f92363d0-718f-4ef6-a9b7-9ca9251c0dc7
  Show excerpt
  [2026-03-14 08:52] xenonfun: ⏺ Subagent working on: 1. Symbol emergence marker — vertical dashed gold line on all charts 2. Coupling sweep comparison table — auto-detect K= runs, show summary 3. Scatter plot — global_r vs code_separat…

Keep researching

Missing something or suspicious of what's here? Kick off a research session — a Claude agent will investigate, cite its sources, and file new facts into a dedicated context you can review before accepting into the shared view.

Code LM

Inbound mentions (12)

branchesToBranches to(1)

comparesCompares(1)

decreasesWithContextSizeDecreases With Context Size(1)

hasComponentHas Component(1)

inputToInput to(1)

isIs(1)

outputsToOutputs to(1)

precedesStagePrecedes Stage(1)

presupposesExistenceOfPresupposes Existence of(1)

sendsDataToSends Data to(1)

stage5Stage5(1)

targetsModelsTargets Models(1)

Other facts (37)

Timeline

References (13)

See also

Keep researching

`branchesTo`Branches to(1)

`compares`Compares(1)

`decreasesWithContextSize`Decreases With Context Size(1)

`hasComponent`Has Component(1)

`inputTo`Input to(1)

`is`Is(1)

`outputsTo`Outputs to(1)

`precedesStage`Precedes Stage(1)

`presupposesExistenceOf`Presupposes Existence of(1)

`sendsDataTo`Sends Data to(1)

`stage5`Stage5(1)

`targetsModels`Targets Models(1)