Training configuration

as donto believed on

From Dontopedia, the open, paraconsistent wiki. (Last updated 2026-06-10.)

Training configuration has 81 facts recorded in Dontopedia across 19 references, with 12 live disagreements.

81 facts·59 predicates·19 sources·12 in dispute

Mostly:rdf:type(8), has batch size(5), has learning rate(2)

Inbound mentions (1)

Other subjects in dontopedia point AT this entity as a value. These are inverse relationships — e.g. "X motherOf this subject" — and answer questions the forward facts can't. Grouped by predicate.

`appliesTo`Applies to(1)

Hyperparameter Tuningex:hyperparameter-tuning

Other facts (79)

The long tail: predicates that appear too rarely to warrant their own section. Filter or scroll to find a specific one. Each row links to its source.

79 facts

Predicate	Value	Ref
Rdf:type	Configuration	[8]
Rdf:type	Configuration	[10]
Rdf:type	Training Configuration	[11]
Rdf:type	Configuration	[12]
Rdf:type	Training Hyperparameter	[15]
Rdf:type	Training Configuration	[16]
Rdf:type	Machine Learning Configuration	[17]
Rdf:type	Training Configuration	[18]
Has Batch Size	12	[1]
Has Batch Size	64	[6]
Has Batch Size	16	[7]
Has Batch Size	16	[13]
Has Batch Size	128	[18]
Has Learning Rate	0.00005	[1]
Has Learning Rate	0.001	[18]
Has Warmup Steps	1000	[1]
Has Warmup Steps	50	[7]
Total Steps	16670	[3]
Total Steps	10000	[11]
Batch Size	4	[3]
Batch Size	64	[11]
Sequence Length	2048	[3]
Sequence Length	256	[11]
Has Seq Len	256	[6]
Has Seq Len	8192	[7]
Has Steps	10000	[6]
Has Steps	1245	[7]
Applied to	Omega	[12]
Applied to	Logits	[12]
Uses	Adam Optimizer	[14]
Uses	Mse Loss	[14]
Has Beta2	0.95	[1]
Has Grad Clip	1	[1]
Has Lr	0.0003	[2]
Has Iters	10000	[2]
Has Warmup	1000	[2]
Save Interval	2000	[3]
Checkpoint Directory	checkpoints/epoch_cl100k	[3]
Log Interval	100	[3]
Warmup Steps	500	[3]
Learning Rate	0.0001	[3]
Validation Interval	2000	[3]
Differs Vocab From First Table	8192 vs 100K	[4]
Has Num Layers	4	[5]
Has D Key	16	[5]
Has D Val	16	[5]
Has G	8	[5]
Has H	2	[5]
Has Num Heads	4	[5]
Causes Improvement in	Dc16 Metric	[6]
Has Lr End	0.0001	[7]
Has Lr Start	0.01	[7]
Has Entity Curriculum Fraction	0.1	[8]
Changed Steps From	100000	[9]
Changed Steps to	200000	[9]
Uses Lr Warmup	true	[12]
Uses Weight Decay	true	[12]
Has Cache Behavior	Cache Growth	[12]
Uses Annealing	true	[12]
Has Sequence Length	256	[13]
Has Model	Score Fusion Model	[14]
Has Optimizer	Adam	[14]
Has Loss Function	Mseloss	[14]
Number of Epochs	10	[14]
Max Epochs	3000	[15]
Has Optimizer	Optimizer	[16]
Has Loss Function	Loss Function	[16]
Has Model	Model Instance	[16]
Has Training Data	Train Dataset	[16]
Has Training Dataset	Train Dataset	[16]
Has Validation Dataset	Val Dataset	[16]
Includes Hyperparameters	Hyperparameter Config	[17]
Includes Guidance	Additional Tips	[17]
Optimization Goal	accuracy-maximization	[17]
Best Model Strategy	load-best-at-end	[17]
Has Accumulation Steps	4	[18]
Has Shuffle	true	[18]
Has Epochs	1	[18]
Includes	Accumulation Parameter	[19]

Timeline axis is valid_time — when each source says the fact was true in the world, not when Dontopedia learned about it. Retracted rows are kept for provenance; coloured stripes indicate the context kind.

81 undated

—

hasLearningRateblah/training-and-evals/part-22

0.00005

—

hasBeta2blah/training-and-evals/part-22

0.95

—

hasGradClipblah/training-and-evals/part-22

—

hasWarmupStepsblah/training-and-evals/part-22

1000

—

hasBatchSizeblah/training-and-evals/part-22

—

hasLrblah/watt-activation/part-84

0.0003

—

hasItersblah/watt-activation/part-84

10000

—

hasWarmupblah/watt-activation/part-84

1000

—

saveIntervalblah/watt-activation/part-127

2000

—

checkpointDirectoryblah/watt-activation/part-127

checkpoints/epoch_cl100k

—

logIntervalblah/watt-activation/part-127

100

—

totalStepsblah/watt-activation/part-127

16670

—

batchSizeblah/watt-activation/part-127

—

warmupStepsblah/watt-activation/part-127

500

—

learningRateblah/watt-activation/part-127

0.0001

—

validationIntervalblah/watt-activation/part-127

2000

—

sequenceLengthblah/watt-activation/part-127

2048

—

differsVocabFromFirstTableblah/watt-activation/part-315

8192 vs 100K

—

hasNumLayersblah/watt-activation/part-607

—

hasDKeyblah/watt-activation/part-607

—

hasDValblah/watt-activation/part-607

—

hasGblah/watt-activation/part-607

—

hasHblah/watt-activation/part-607

—

hasNumHeadsblah/watt-activation/part-607

—

causesImprovementInblah/watt-activation/part-354

ex:dc16-metric

—

hasSeqLenblah/watt-activation/part-354

256

—

hasBatchSizeblah/watt-activation/part-354

—

hasStepsblah/watt-activation/part-354

10000

—

hasWarmupStepsblah/watt-activation/part-420

—

hasBatchSizeblah/watt-activation/part-420

—

hasLrEndblah/watt-activation/part-420

0.0001

—

hasLrStartblah/watt-activation/part-420

0.01

—

hasSeqLenblah/watt-activation/part-420

8192

—

hasStepsblah/watt-activation/part-420

1245

—

typeblah/random/38

ex:Configuration

—

hasEntityCurriculumFractionblah/random/38

0.1

—

changedStepsFromblah/vidya/6

100000

—

changedStepsToblah/vidya/6

200000

—

typeblah/watt-activation/84

ex:Configuration

—

typeblah/watt-activation/352

ex:TrainingConfiguration

—

totalStepsblah/watt-activation/352

10000

—

batchSizeblah/watt-activation/352

—

sequenceLengthblah/watt-activation/352

256

—

typeblah/watt-activation/479

ex:Configuration

—

usesLrWarmupblah/watt-activation/479

true

—

usesWeightDecayblah/watt-activation/479

true

—

appliedToblah/watt-activation/479

ex:omega

—

hasCacheBehaviorblah/watt-activation/479

ex:cache-growth

—

usesAnnealingblah/watt-activation/479

true

—

appliedToblah/watt-activation/479

ex:logits

—

hasBatchSizeblah/watt-activation/670

—

hasSequenceLengthblah/watt-activation/670

256

—

hasModelbeam/4b0fb0ca-8535-46e3-955c-5f7eb8b91c01

ex:score-fusion-model

—

hasOptimizerbeam/4b0fb0ca-8535-46e3-955c-5f7eb8b91c01

ex:adam

—

hasLossFunctionbeam/4b0fb0ca-8535-46e3-955c-5f7eb8b91c01

ex:mseloss

—

numberOfEpochsbeam/4b0fb0ca-8535-46e3-955c-5f7eb8b91c01

—

usesbeam/4b0fb0ca-8535-46e3-955c-5f7eb8b91c01

ex:Adam-optimizer

—

usesbeam/4b0fb0ca-8535-46e3-955c-5f7eb8b91c01

ex:MSELoss

—

maxEpochsbeam/06eb4544-0695-497b-a79a-f7602f0d8ecc

3000

—

typebeam/06eb4544-0695-497b-a79a-f7602f0d8ecc

ex:training-hyperparameter

—

typebeam/2739fb08-c4fc-4bb6-b143-e05bc2133eae

ex:TrainingConfiguration

—

has-optimizerbeam/2739fb08-c4fc-4bb6-b143-e05bc2133eae

ex:optimizer

—

has-loss-functionbeam/2739fb08-c4fc-4bb6-b143-e05bc2133eae

ex:loss-function

—

has-modelbeam/2739fb08-c4fc-4bb6-b143-e05bc2133eae

ex:model-instance

—

has-training-databeam/2739fb08-c4fc-4bb6-b143-e05bc2133eae

ex:train-dataset

—

has-training-datasetbeam/2739fb08-c4fc-4bb6-b143-e05bc2133eae

ex:train-dataset

—

has-validation-datasetbeam/2739fb08-c4fc-4bb6-b143-e05bc2133eae

ex:val-dataset

—

typebeam/cce29709-18fd-476c-8bcc-de705b470912

ex:MachineLearningConfiguration

—

labelbeam/cce29709-18fd-476c-8bcc-de705b470912

Model Training Configuration

—

includesHyperparametersbeam/cce29709-18fd-476c-8bcc-de705b470912

ex:hyperparameter-config

—

includesGuidancebeam/cce29709-18fd-476c-8bcc-de705b470912

ex:additional-tips

—

optimizationGoalbeam/cce29709-18fd-476c-8bcc-de705b470912

accuracy-maximization

—

bestModelStrategybeam/cce29709-18fd-476c-8bcc-de705b470912

load-best-at-end

—

typebeam/0a6354af-a6f7-4051-8cb3-e50345232784

ex:TrainingConfiguration

—

labelbeam/0a6354af-a6f7-4051-8cb3-e50345232784

Training configuration

—

hasBatchSizebeam/0a6354af-a6f7-4051-8cb3-e50345232784

128

—

hasLearningRatebeam/0a6354af-a6f7-4051-8cb3-e50345232784

0.001

—

hasAccumulationStepsbeam/0a6354af-a6f7-4051-8cb3-e50345232784

—

hasShufflebeam/0a6354af-a6f7-4051-8cb3-e50345232784

true

—

hasEpochsbeam/0a6354af-a6f7-4051-8cb3-e50345232784

—

includesbeam/8b6abd69-54a1-41b8-bb85-d0b80bff1a3a

ex:accumulation-parameter

References (19)

19 references

[1]Part 225 facts
ctx:discord/blah/training-and-evals/part-22
[2]Part 843 facts
ctx:discord/blah/watt-activation/part-84
[3]Part 1279 facts
ctx:discord/blah/watt-activation/part-127
[4]Part 3151 fact
ctx:discord/blah/watt-activation/part-315
[5]Part 6076 facts
ctx:discord/blah/watt-activation/part-607
[6]Part 3544 facts
ctx:discord/blah/watt-activation/part-354
[7]Part 4206 facts
ctx:discord/blah/watt-activation/part-420
[8]382 facts
ctx:discord/blah/random/38
- full textrandom-38
  text/plain2 KBdoc:agent/random-38/b1cacc60-bd67-4179-b775-c64827aa1d57
  Show excerpt
  [2026-03-19 00:19] xenonfun: ``` ⏺ PD-HAM at step 15K — the protect gate has not specialized. - protect_gate = 0.506 (same ~50% as step 6K — no movement) - eff_gate = 0.278 (same) - write_fraction = 1.0 (every token still writes) -…
[9]62 facts
ctx:discord/blah/vidya/6
- full textvidya-6
  text/plain3 KBdoc:agent/vidya-6/cda90ecf-8302-448a-a889-53b5a677fef3
  Show excerpt
  [2026-02-21 10:36] rolandnsharp7643: >so what did we complete today. we added reinforcement learning. and changed the data set and what else …
[10]841 fact
ctx:discord/blah/watt-activation/84
- full textwatt-activation-84
  text/plain3 KBdoc:agent/watt-activation-84/16e41088-c84d-4a6f-9c2d-56d69830cfa6
  Show excerpt
  [2026-03-07 20:41] xenonfun: okay some instant issues with this much data: ``` The problem: mx.eval(loss, model.parameters(), optimizer.state) traverses the full tree of 113M params + Adam's 2x state every step. For the compiled path, mx.ev…
[11]3524 facts
ctx:discord/blah/watt-activation/352
- full textwatt-activation-352
  text/plain2 KBdoc:agent/watt-activation-352/f9fe3319-d5f4-4e70-b415-d397928b4c05
  Show excerpt
  [2026-03-17 06:32] xenonfun: ``` 44 +├── antenna.py # AntennaHarmonicBlock + AntennaLM: field-mediated byte LM 45 +├── antenna_probes.py # Diagnostic probes: impulse, memory, coupling, leakage, boundary 46 +├── an…
[12]4797 facts
ctx:discord/blah/watt-activation/479
- full textwatt-activation-479
  text/plain2 KBdoc:agent/watt-activation-479/cf877f60-5f22-46ef-a130-a278610bc58d
  Show excerpt
  [2026-03-21 23:05] xenonfun: ``` ⏺ All committed and pushed. Server is live at http://localhost:42069/ with full controls. Final session stats: ┌────────────────────┬──────────────────┬───────────────────────────────┐ │ Metric…
[13]6702 facts
ctx:discord/blah/watt-activation/670
- full textwatt-activation-670
  text/plain3 KBdoc:agent/watt-activation-670/d9fd63e9-d1a4-4d2d-9849-fcaa1f434b61
  Show excerpt
  [2026-04-20 17:11] xenonfun: Important observations: 1. Neither feedback variant is catastrophically diverging at peak LR 3e-3. The model produces grammatically-shaped output; the damage is only at the vocabulary level, not structural.…
[14]4b0fb0ca 8535 46e3 955c 5f7eb8b91c016 facts
ctx:claims/beam/4b0fb0ca-8535-46e3-955c-5f7eb8b91c01
[15]06eb4544 0695 497b A79a F7602f0d8ecc2 facts
ctx:claims/beam/06eb4544-0695-497b-a79a-f7602f0d8ecc
- full textbeam-chunk
  text/plain1 KBdoc:beam/06eb4544-0695-497b-a79a-f7602f0d8ecc
  Show excerpt
  print(f"Early stopping triggered at epoch {epoch}") break print(f"Epoch {epoch+1}/{3000}, Training Loss: {loss.item():.4f}, Validation Loss: {avg_val_loss:.4f}") # Save the model torch.save(model.state_dict(), …
[16]2739fb08 C4fc 4bb6 B143 E05bc2133eae7 facts
ctx:claims/beam/2739fb08-c4fc-4bb6-b143-e05bc2133eae
- full textbeam-chunk
  text/plain1 KBdoc:beam/2739fb08-c4fc-4bb6-b143-e05bc2133eae
  Show excerpt
```python import torch import torch.nn as nn import torch.optim as optim from torch.utils.data import DataLoader, TensorDataset from sklearn.model_selection import train_test_split from sklearn.metrics import mean_squared_error class MyMod…
[17]Cce29709 18fd 476c 8bcc De705b4709126 facts
ctx:claims/beam/cce29709-18fd-476c-8bcc-de705b470912
- full textbeam-chunk
  text/plain1 KBdoc:beam/cce29709-18fd-476c-8bcc-de705b470912
  Show excerpt
  logging_steps=10, evaluation_strategy='epoch', save_strategy='epoch', load_best_model_at_end=True, metric_for_best_model='accuracy', learning_rate=2e-5, ) ``` ### Additional Tips - **Experimentation**: Start with t…
[18]0a6354af A6f7 4051 8cb3 E503452327847 facts
ctx:claims/beam/0a6354af-a6f7-4051-8cb3-e50345232784
[19]8b6abd69 54a1 41b8 Bb85 D0b80bff1a3a1 fact
ctx:claims/beam/8b6abd69-54a1-41b8-bb85-d0b80bff1a3a
- full textbeam-chunk
  text/plain1 KBdoc:beam/8b6abd69-54a1-41b8-bb85-d0b80bff1a3a
  Show excerpt
  loss = criterion(outputs, batch_targets) # Normalize the loss because it is accumulated loss = loss / accumulation_steps # Backward pass loss.backward() # Update wei…

Keep researching

Missing something or suspicious of what's here? Kick off a research session — a Claude agent will investigate, cite its sources, and file new facts into a dedicated context you can review before accepting into the shared view.

Training configuration

Inbound mentions (1)

appliesToApplies to(1)

Other facts (79)

Timeline

References (19)

See also

Keep researching

`appliesTo`Applies to(1)