Training Run Config
From Dontopedia, the open, paraconsistent wiki. (Last updated 2026-06-06.)
Training Run Config has 21 facts recorded in Dontopedia across 3 references, with 3 live disagreements.
Mostly:batch size(2), sequence length(2), learning rate(2)
Maturity scale
raw canonical shape-checked rule-derived certifiedInbound mentions (2)
Other subjects in dontopedia point AT this entity as a value. These are inverse relationships — e.g. "X motherOf this subject" — and answer questions the forward facts can't. Grouped by predicate.
requestedRunRequested Run(1)
- Xenonfun
ex:xenonfun
usesConfigurationUses Configuration(1)
- Training Process
ex:training-process
Other facts (21)
The long tail: predicates that appear too rarely to warrant their own section. Filter or scroll to find a specific one. Each row links to its source.
| Predicate | Value | Ref |
|---|---|---|
| Batch Size | 4 | [1] |
| Batch Size | 64 | [3] |
| Sequence Length | 2048 | [1] |
| Sequence Length | 8000 | [3] |
| Learning Rate | 0.0001 | [1] |
| Learning Rate | 0.000008 | [2] |
| Warmup Steps | 500 | [1] |
| Save Frequency | 2000 | [1] |
| Validation Frequency | 2000 | [1] |
| Log Frequency | 100 | [1] |
| Checkpoint Directory | checkpoints/epoch_cl100k | [1] |
| Rdf:type | Training Configuration | [2] |
| Step Count | 6000 | [2] |
| Throughput Rate | 12000 | [2] |
| Calculated Tokens Seen | 82000000 | [2] |
| Data Type | Instruction Data | [2] |
| Learning Rate Decay Schedule | cosine decay | [2] |
| Initial Learning Rate | 0.00002 | [2] |
| Current Epoch | 2 | [2] |
| Iterations | 100 | [3] |
| Applies to | Both Configs | [3] |
Timeline
Timeline axis is valid_time — when each source says the fact was true in the world, not when Dontopedia learned about it. Retracted rows are kept for provenance; coloured stripes indicate the context kind.
References (3)
ctx:discord/blah/watt-activation/139- full textwatt-activation-139text/plain2 KB
doc:agent/watt-activation-139/84b665a8-03e4-433b-97dc-f40068180b02Show excerpt
[2026-03-09 07:23] xenonfun: ⏺ Training is resumed from step 10000. The NaN is fixed — all three patches held: 1. r_global = mx.sqrt(sum_sq + 1e-8) — eps inside sqrt for order parameter 2. mean_spec / mx.sqrt(sum_sq + 1e-8) — eps insid…
ctx:discord/blah/watt-activation/169- full textwatt-activation-169text/plain3 KB
doc:agent/watt-activation-169/41f410cf-a551-436e-a024-061c08e0e3efShow excerpt
[2026-03-09 20:11] xenonfun: ⏺ Still confused — it sees "Write a short poem" and almost gets there ("He was a large time... It was a dream of a song") but then it mixes in code and FineWeb patterns. The model knows the instruction format …
ctx:discord/blah/watt-activation/408- full textwatt-activation-408text/plain2 KB
doc:agent/watt-activation-408/5084227c-09bf-490d-affa-7c62f4061de3Show excerpt
[2026-03-19 06:38] xenonfun: ⏺ Yes — the same way we did decay gates. The coupling strength should be derived from the oscillator state, not fixed: Principled approach: coupling = f(coherence between the two groups being coupled). Groups…
See also
Keep researching
Missing something or suspicious of what's here? Kick off a research session — a Claude agent will investigate, cite its sources, and file new facts into a dedicated context you can review before accepting into the shared view.