Training Run 1

as donto believed on

From Dontopedia, the open, paraconsistent wiki. (Last updated 2026-06-06.)

Training Run 1 has 323 facts recorded in Dontopedia across 18 references, with 35 live disagreements.

323 facts·223 predicates·18 sources·35 in dispute

Mostly:has progress step(16), logged metric(10), has summary metric(10)

Has Progress Stepin dispute`hasProgressStep`

2100^[8]sourceall time · 153
2200^[8]sourceall time · 153
2300^[8]sourceall time · 153
2400^[8]sourceall time · 153
2500^[8]sourceall time · 153
2600^[8]sourceall time · 153
2700^[8]sourceall time · 153
2800^[8]sourceall time · 153
2900^[8]sourceall time · 153
3000^[8]sourceall time · 153

Logged Metricin dispute`loggedMetric`

gpu/active_gb^[12]sourceall time · 238
gpu/cache_gb^[12]sourceall time · 238
gpu/mem_pct^[12]sourceall time · 238
gpu/step_peak_gb^[12]sourceall time · 238
gpu/util_pct^[12]sourceall time · 238
train/b00_K^[12]sourceall time · 238
train/b00_b^[12]sourceall time · 238
train/b00_bg^[12]sourceall time · 238
train/b00_mp0^[12]sourceall time · 238
train/b00_mp1^[12]sourceall time · 238

Has Summary Metricin dispute`hasSummaryMetric`

gpu/active_gb^[12]sourceall time · 238
gpu/cache_gb^[12]sourceall time · 238
gpu/mem_pct^[12]sourceall time · 238
gpu/step_peak_gb^[12]sourceall time · 238
gpu/util_pct^[12]sourceall time · 238
train/b00_K^[12]sourceall time · 238
train/b00_b^[12]sourceall time · 238
train/b00_bg^[12]sourceall time · 238
train/b00_mp0^[12]sourceall time · 238
train/b00_mp1^[12]sourceall time · 238

Has Summary Valuein dispute`hasSummaryValue`

0.264^[12]sourceall time · 238
7.374^[12]sourceall time · 238
7.74^[12]sourceall time · 238
6.464^[12]sourceall time · 238
98^[12]sourceall time · 238
0.1751^[12]sourceall time · 238
28.1559^[12]sourceall time · 238
0.1065^[12]sourceall time · 238
0.2559^[12]sourceall time · 238
0.1304^[12]sourceall time · 238

Inbound mentions (9)

Other subjects in dontopedia point AT this entity as a value. These are inverse relationships — e.g. "X motherOf this subject" — and answer questions the forward facts can't. Grouped by predicate.

`claimsAboutSubject`Claims About Subject(1)

Message 2ex:message-2

`containsLogOutput`Contains Log Output(1)

Log Message 2026 03 09 18 04ex:log-message-2026-03-09-18-04

`expectsTweaksBeforeHf`Expects Tweaks Before Hf(1)

Traves Thebergeex:traves-theberge

`follows`Follows(1)

Evaluation Event 10000ex:evaluation-event-10000

`partOfTrainingRun`Part of Training Run(1)

Iteration 10ex:iteration-10

`possessesLogsMetrics`Possesses Logs Metrics(1)

Traves Thebergeex:traves-theberge

`presentsLogData`Presents Log Data(1)

Utterance 2ex:utterance-2

`runsOnCpu`Runs on Cpu(1)

Traves Thebergeex:traves-theberge

`targetOfModification`Target of Modification(1)

Training Loopex:training-loop

Other facts (274)

The long tail: predicates that appear too rarely to warrant their own section. Filter or scroll to find a specific one. Each row links to its source.

274 facts

Predicate	Value	Ref
Total Steps	1000	[4]
Total Steps	16684	[9]
Total Steps	22920	[10]
Total Steps	12432	[12]
Total Steps	20000	[13]
Total Steps	1245	[15]
Total Steps	1245	[16]
Rdf:type	Training Iteration	[5]
Rdf:type	Training Run	[7]
Rdf:type	Model Training	[12]
Rdf:type	Training Run	[15]
Rdf:type	Machine Learning Training Run	[16]
Rdf:type	Training Run	[17]
Rdf:type	Model Training Run	[18]
Has Perplexity	11.4	[2]
Has Perplexity	11.6	[2]
Has Perplexity	48.06	[5]
Has Perplexity	175.5	[6]
Has Perplexity	176	[10]
Has Perplexity	96.9	[12]
Has Learning Rate	0.00003	[2]
Has Learning Rate	0.0003	[5]
Has Learning Rate	0.000019	[10]
Has Learning Rate	0.0000142	[12]
Has Estimated Time Remaining	304	[2]
Has Estimated Time Remaining	287	[2]
Has Estimated Time Remaining	22m 45s	[5]
Has Estimated Time Remaining	6min	[12]
Current Step	3500	[10]
Current Step	10775	[12]
Current Step	3415	[13]
Current Step	1200	[18]
Has Final Validation Loss	2.4532	[2]
Has Final Validation Loss	4.9257	[11]
Has Final Validation Loss	4.5679	[12]
Has Status	gone	[3]
Has Status	complete	[7]
Has Status	Slowly Improving	[14]
Has Metric	0.759	[6]
Has Metric	0.423	[6]
Has Metric	3.47	[6]
Estimated Time Remaining	216min	[10]
Estimated Time Remaining	92min	[13]
Estimated Time Remaining	13 min	[14]
Has Iteration	830	[2]
Has Iteration	840	[2]
Has Loss at Iteration	2.4261	[2]
Has Loss at Iteration	2.4179	[2]
Has Validation Loss	2.4313	[2]
Has Validation Loss	2.4551	[2]
Has Learning Rate at Iter	0.000013	[2]
Has Learning Rate at Iter	0.000012	[2]
Has Best Loss	2.3954	[2]
Has Best Loss	3.0441	[5]
Has Elapsed Time	271	[2]
Has Elapsed Time	293	[2]
Has Patience	9	[2]
Has Patience	10	[2]
Has Total Iterations	50000	[5]
Has Total Iterations	20000	[6]
Has Iteration Speed	29.3	[5]
Has Iteration Speed	27	[6]
Has Token Speed	30000	[5]
Has Token Speed	55800	[6]
Total Tokens Processed	819200	[7]
Total Tokens Processed	203685888	[12]
Average Tokens Per Second	6750	[7]
Average Tokens Per Second	73085	[12]
Has Loss Value	5.1705	[10]
Has Loss Value	4.5741	[12]
Has Final Perplexity	137.8	[11]
Has Final Perplexity	96.3	[12]
Has New Best Perplexity	137.8	[11]
Has New Best Perplexity	96.3	[12]
Step Time	212	[12]
Step Time	332	[13]
Throughput	77270	[12]
Throughput	98782	[13]
Progress Percentage	0.171	[13]
Progress Percentage	2	[15]
Learning Rate	0.0000943	[13]
Learning Rate	0.005	[15]
Batch Size	64	[13]
Batch Size	64	[16]
Sequence Length	512	[13]
Sequence Length	8192	[16]
On Cpu	Cpu	[1]
Has Batch Size	64	[2]
Has Decayed Learning Rate	0.000003	[2]
Has Best Loss Iteration	821	[2]
Has Iteration Time	1802	[2]
Has Max Patience	10	[2]
Has Best Validation Loss	2.4295	[2]
Saved Checkpoint	checkpoint.bin	[2]
Has Total Time	295.7	[2]
Has Average Iteration Time	1802	[2]
Has Final Train Loss	2.3986	[2]
Has Initial Train Loss	2.4532	[2]
Has Initial Validation Loss	2.4435	[2]
Estimated Duration	~8.5 hours	[4]

Timeline axis is valid_time — when each source says the fact was true in the world, not when Dontopedia learned about it. Retracted rows are kept for provenance; coloured stripes indicate the context kind.

323 undated

—

onCpublah/training-and-evals/part-2

ex:cpu

—

hasBatchSizeblah/random/24

—

hasLearningRateblah/random/24

0.00003

—

hasDecayedLearningRateblah/random/24

0.000003

—

hasIterationblah/random/24

830

—

hasLossAtIterationblah/random/24

2.4261

—

hasValidationLossblah/random/24

2.4313

—

hasPerplexityblah/random/24

11.4

—

hasLearningRateAtIterblah/random/24

0.000013

—

hasBestLossblah/random/24

2.3954

—

hasBestLossIterationblah/random/24

821

—

hasIterationTimeblah/random/24

1802

—

hasElapsedTimeblah/random/24

271

—

hasEstimatedTimeRemainingblah/random/24

304

—

hasPatienceblah/random/24

—

hasMaxPatienceblah/random/24

—

hasIterationblah/random/24

840

—

hasLossAtIterationblah/random/24

2.4179

—

hasValidationLossblah/random/24

2.4551

—

hasPerplexityblah/random/24

11.6

—

hasLearningRateAtIterblah/random/24

0.000012

—

hasElapsedTimeblah/random/24

293

—

hasEstimatedTimeRemainingblah/random/24

287

—

hasPatienceblah/random/24

—

hasBestValidationLossblah/random/24

2.4295

—

savedCheckpointblah/random/24

checkpoint.bin

—

hasTotalTimeblah/random/24

295.7

—

hasAverageIterationTimeblah/random/24

1802

—

hasFinalTrainLossblah/random/24

2.3986

—

hasInitialTrainLossblah/random/24

2.4532

—

hasFinalValidationLossblah/random/24

2.4532

—

hasInitialValidationLossblah/random/24

2.4435

—

labelblah/training-and-evals/38

previous training run

—

hasStatusblah/training-and-evals/38

gone

—

totalStepsblah/watt-activation/13

1000

—

estimatedDurationblah/watt-activation/13

~8.5 hours

—

typeblah/watt-activation/20

ex:TrainingIteration

—

hasIterationNumberblah/watt-activation/20

10000

—

hasTotalIterationsblah/watt-activation/20

50000

—

hasLossblah/watt-activation/20

3.8724

—

hasPerplexityblah/watt-activation/20

48.06

—

hasAverageLossOverLast100blah/watt-activation/20

3.9294

—

hasAveragePerplexityOverLast100blah/watt-activation/20

50.88

—

hasBestLossblah/watt-activation/20

3.0441

—

hasLearningRateblah/watt-activation/20

0.0003

—

hasIterationSpeedblah/watt-activation/20

29.3

—

hasTokenSpeedblah/watt-activation/20

30000

—

hasEstimatedTimeRemainingblah/watt-activation/20

22m 45s

—

followsblah/watt-activation/20

ex:log-entry-2026-03-06-05-09

—

usesConfigblah/watt-activation/61

ex:config-anchor-v3-m32-l2048

—

hasIterationCountblah/watt-activation/61

2000

—

hasTotalIterationsblah/watt-activation/61

20000

—

hasAverageLossblah/watt-activation/61

5.1675

—

hasPerplexityblah/watt-activation/61

175.5

—

hasMetricblah/watt-activation/61

0.759

—

hasMetricblah/watt-activation/61

0.423

—

hasClusterCountblah/watt-activation/61

—

hasMetricblah/watt-activation/61

3.47

—

hasEffectiveAnchorsblah/watt-activation/61

—

hasIterationSpeedblah/watt-activation/61

—

hasTokenSpeedblah/watt-activation/61

55800

—

labelblah/watt-activation/114

Training Run 1

—

typeblah/watt-activation/114

ex:TrainingRun

—

loadedSequenceCountblah/watt-activation/114

5000

—

loadedSequenceLengthblah/watt-activation/114

2048

—

trainsModelblah/watt-activation/114

ex:harmonicgpt

—

hasStatusblah/watt-activation/114

complete

—

finalLossblah/watt-activation/114

3.1939

—

elapsedTimeSecondsblah/watt-activation/114

121.4

—

totalTokensProcessedblah/watt-activation/114

819200

—

averageTokensPerSecondblah/watt-activation/114

6750

—

gpuPeakMemoryGBblah/watt-activation/114

34.31

—

gpuActiveMemoryGBblah/watt-activation/114

1.01

—

gpuCacheMemoryGBblah/watt-activation/114

34.75

—

processResidentSetSizeGBblah/watt-activation/114

4.16

—

precededByblah/watt-activation/114

ex:utterance-1

—

hasProgressStepblah/watt-activation/153

2100

—

hasTotalStepsblah/watt-activation/153

16684

—

step2100Lossblah/watt-activation/153

6.0128

—

step2100Perplexityblah/watt-activation/153

408.6

—

step2100LearningRateblah/watt-activation/153

0.0000487

—

step2100StepDurationblah/watt-activation/153

668

—

step2100Throughputblah/watt-activation/153

12271

—

step2100EstimatedTimeRemainingblah/watt-activation/153

162min

—

hasProgressStepblah/watt-activation/153

2200

—

step2200Lossblah/watt-activation/153

6.0229

—

step2200Perplexityblah/watt-activation/153

412.8

—

step2200LearningRateblah/watt-activation/153

0.0000485

—

step2200StepDurationblah/watt-activation/153

665

—

step2200Throughputblah/watt-activation/153

12326

—

step2200EstimatedTimeRemainingblah/watt-activation/153

160min

—

hasProgressStepblah/watt-activation/153

2300

—

step2300Lossblah/watt-activation/153

6.0173

—

step2300Perplexityblah/watt-activation/153

410.5

—

step2300LearningRateblah/watt-activation/153

0.0000484

—

step2300StepDurationblah/watt-activation/153

665

—

step2300Throughputblah/watt-activation/153

12320

—

step2300EstimatedTimeRemainingblah/watt-activation/153

159min

—

hasProgressStepblah/watt-activation/153

2400

—

step2400Lossblah/watt-activation/153

5.9659

—

step2400Perplexityblah/watt-activation/153

389.9

—

step2400LearningRateblah/watt-activation/153

0.0000482

—

step2400StepDurationblah/watt-activation/153

666

—

step2400Throughputblah/watt-activation/153

12308

—

step2400EstimatedTimeRemainingblah/watt-activation/153

158min

—

hasProgressStepblah/watt-activation/153

2500

—

step2500Lossblah/watt-activation/153

5.9942

—

step2500Perplexityblah/watt-activation/153

401.1

—

step2500LearningRateblah/watt-activation/153

0.000048

—

step2500StepDurationblah/watt-activation/153

667

—

step2500Throughputblah/watt-activation/153

12290

—

step2500EstimatedTimeRemainingblah/watt-activation/153

158min

—

hasProgressStepblah/watt-activation/153

2600

—

step2600Lossblah/watt-activation/153

6.0035

—

step2600Perplexityblah/watt-activation/153

404.9

—

step2600LearningRateblah/watt-activation/153

0.0000478

—

step2600StepDurationblah/watt-activation/153

674

—

step2600Throughputblah/watt-activation/153

12161

—

step2600EstimatedTimeRemainingblah/watt-activation/153

158min

—

hasProgressStepblah/watt-activation/153

2700

—

step2700Lossblah/watt-activation/153

6.0127

—

step2700Perplexityblah/watt-activation/153

408.6

—

step2700LearningRateblah/watt-activation/153

0.0000477

—

step2700StepDurationblah/watt-activation/153

670

—

step2700Throughputblah/watt-activation/153

12220

—

step2700EstimatedTimeRemainingblah/watt-activation/153

156min

—

hasProgressStepblah/watt-activation/153

2800

—

step2800Lossblah/watt-activation/153

5.969

—

step2800Perplexityblah/watt-activation/153

391.1

—

step2800LearningRateblah/watt-activation/153

0.0000475

—

step2800StepDurationblah/watt-activation/153

674

—

step2800Throughputblah/watt-activation/153

12162

—

step2800EstimatedTimeRemainingblah/watt-activation/153

156min

—

hasProgressStepblah/watt-activation/153

2900

—

step2900Lossblah/watt-activation/153

6.0019

—

step2900Perplexityblah/watt-activation/153

404.2

—

step2900LearningRateblah/watt-activation/153

0.0000473

—

step2900StepDurationblah/watt-activation/153

678

—

step2900Throughputblah/watt-activation/153

12083

—

step2900EstimatedTimeRemainingblah/watt-activation/153

156min

—

hasProgressStepblah/watt-activation/153

3000

—

step3000Lossblah/watt-activation/153

5.9616

—

step3000Perplexityblah/watt-activation/153

388.2

—

step3000LearningRateblah/watt-activation/153

0.0000471

—

step3000StepDurationblah/watt-activation/153

674

—

step3000Throughputblah/watt-activation/153

12150

—

step3000EstimatedTimeRemainingblah/watt-activation/153

154min

—

hasProgressStepblah/watt-activation/153

3100

—

step3100Lossblah/watt-activation/153

5.9819

—

step3100Perplexityblah/watt-activation/153

396.2

—

step3100LearningRateblah/watt-activation/153

0.0000468

—

step3100StepDurationblah/watt-activation/153

680

—

step3100Throughputblah/watt-activation/153

12047

—

step3100EstimatedTimeRemainingblah/watt-activation/153

154min

—

hasProgressStepblah/watt-activation/153

3200

—

step3200Lossblah/watt-activation/153

5.9817

—

step3200Perplexityblah/watt-activation/153

396.1

—

step3200LearningRateblah/watt-activation/153

0.0000466

—

step3200StepDurationblah/watt-activation/153

678

—

step3200Throughputblah/watt-activation/153

12091

—

step3200EstimatedTimeRemainingblah/watt-activation/153

152min

—

hasProgressStepblah/watt-activation/153

3300

—

step3300Lossblah/watt-activation/153

5.9795

—

step3300Perplexityblah/watt-activation/153

395.3

—

step3300LearningRateblah/watt-activation/153

0.0000464

—

step3300StepDurationblah/watt-activation/153

666

—

step3300Throughputblah/watt-activation/153

12296

—

step3300EstimatedTimeRemainingblah/watt-activation/153

149min

—

hasProgressStepblah/watt-activation/153

3400

—

step3400Lossblah/watt-activation/153

6.0046

—

step3400Perplexityblah/watt-activation/153

405.3

—

step3400LearningRateblah/watt-activation/153

0.0000461

—

step3400StepDurationblah/watt-activation/153

716

—

step3400Throughputblah/watt-activation/153

11435

—

step3400EstimatedTimeRemainingblah/watt-activation/153

159min

—

hasProgressStepblah/watt-activation/153

3500

—

step3500Lossblah/watt-activation/153

5.994

—

step3500Perplexityblah/watt-activation/153

401

—

step3500LearningRateblah/watt-activation/153

0.0000459

—

step3500StepDurationblah/watt-activation/153

674

—

step3500Throughputblah/watt-activation/153

12159

—

step3500EstimatedTimeRemainingblah/watt-activation/153

148min

—

hasProgressStepblah/watt-activation/153

4000

—

step4000Lossblah/watt-activation/153

6.0023

—

step4000Perplexityblah/watt-activation/153

404.4

—

step4000LearningRateblah/watt-activation/153

0.0000446

—

step4000StepDurationblah/watt-activation/153

667

—

step4000Throughputblah/watt-activation/153

12288

—

step4000EstimatedTimeRemainingblah/watt-activation/153

141min

—

totalStepsblah/watt-activation/160

16684

—

currentStepblah/watt-activation/168

3500

—

totalStepsblah/watt-activation/168

22920

—

currentEpochblah/watt-activation/168

—

totalEpochsblah/watt-activation/168

—

epochProgressPercentageblah/watt-activation/168

—

hasLossValueblah/watt-activation/168

5.1705

—

hasPerplexityblah/watt-activation/168

176

—

hasLearningRateblah/watt-activation/168

0.000019

—

stepDurationblah/watt-activation/168

668

—

throughputTokensPerSecondblah/watt-activation/168

12260

References (18)

18 references

[1]Part 21 fact
ctx:discord/blah/training-and-evals/part-2
[2]2431 facts
ctx:discord/blah/random/24
- full textrandom-24
  text/plain3 KBdoc:agent/random-24/c8401fc9-2a91-4ebc-833d-8d389b917d0e
  Show excerpt
  [2026-02-17 05:08] xenonfun: well I'm at minutes now, so I got better output ``` Batch size: 64 Learning rate: 0.00003 → 0.000003 Iter 830 | Loss: 2.4261 | Val: 2.4313 (ppl 11.4) | LR: 0.000013 | Best: 2.3954 @821 | 1802ms/iter | 271s ela…
[3]382 facts
ctx:discord/blah/training-and-evals/38
- full texttraining-and-evals-38
  text/plain3 KBdoc:agent/training-and-evals-38/63e5dd0a-e512-4eac-99ca-888173db5bcc
  Show excerpt
  [2026-03-09 12:03] foxhop.: (files: Screenshot_from_2026-03-09_08-01-18.png, Screenshot_from_2026-03-09_08-01-13.png) [2026-03-09 12:04] foxhop.: We are COOOOOOOOOOKING [2026-03-09 12:04] foxhop.: (files: Screenshot_from_2026-03-09_08-04-…
[4]132 facts
ctx:discord/blah/watt-activation/13
- full textwatt-activation-13
  text/plain2 KBdoc:agent/watt-activation-13/8fb2ebd5-d74a-4511-9dac-1a7047d803b2
  Show excerpt
  [2026-02-28 18:28] xenonfun: 2026-02-28 13:23:39,591 [INFO] cross_species_1b.finetune: Step 10/1000: train_loss=7.9112, lr=2.20e-06, 1051 tok/s **1000 steps ≈ 8.5 hours.** [2026-02-28 18:31] xenonfun: (files: Screenshot_2026-02-28_at_…
[5]2013 facts
ctx:discord/blah/watt-activation/20
- full textwatt-activation-20
  text/plain3 KBdoc:agent/watt-activation-20/3112a2c6-bfc5-4e04-a46c-ccbafb3ff570
  Show excerpt
  [2026-03-06 05:09] xenonfun: Good picture now. Three concrete wins: 1. Lazy eval overlap — MLX compiled step is lazy until mx.eval. We can prepare the next batch on CPU while the GPU is executing, completely free. 2. Flash attention — …
[6]6112 facts
ctx:discord/blah/watt-activation/61
- full textwatt-activation-61
  text/plain3 KBdoc:agent/watt-activation-61/3d9a288b-f7ca-4227-8d92-0bca73a33496
  Show excerpt
  [2026-03-07 10:25] xenonfun: ``` Config: anchor_v3_m32_L2048 (seq_len=2048, kwargs={'use_anchor': True, 'n_anchors': 32}) …
[7]11415 facts
ctx:discord/blah/watt-activation/114
- full textwatt-activation-114
  text/plain2 KBdoc:agent/watt-activation-114/bc8b5ab5-bbc4-4aea-ac89-9211a96127c3
  Show excerpt
  [2026-03-08 22:42] xenonfun: Now add val evaluation in the training loop, after the checkpoint save block [2026-03-08 22:44] xenonfun: ``` === linear (seq=2048) === Training HarmonicGPT | attn=linear d=768 L=12 H=12 seq=2048 Parameters: 86,…
[8]153113 facts
ctx:discord/blah/watt-activation/153
- full textwatt-activation-153
  text/plain3 KBdoc:agent/watt-activation-153/adeb3de4-7496-4fe9-a51b-b54f859c8ef4
  Show excerpt
  [2026-03-09 16:00] xenonfun: step 2100/16684 12.6% loss=6.0128 ppl= 408.6 lr=4.87e-05 668ms 12,271tok/s eta=162min step 2200/16684 13.2% loss=6.0229 ppl= 412.8 lr=4.85e-05 665ms 12,326tok/s eta=160min step 2300/16684 …
[9]1601 fact
ctx:discord/blah/watt-activation/160
- full textwatt-activation-160
  text/plain2 KBdoc:agent/watt-activation-160/83f1326e-5f35-47f2-901e-bb1cc61a1eae
  Show excerpt
  [2026-03-09 16:37] xenonfun: ⏺ With dropout=0.0, the dropout layers are no-ops — they pass through unchanged during both training and inference. So model.train(False) vs model.train(True) makes no difference for our current config. It …
[10]16811 facts
ctx:discord/blah/watt-activation/168
- full textwatt-activation-168
  text/plain3 KBdoc:agent/watt-activation-168/73ee12a6-c466-46d0-8fcc-aeb7b6f8614e
  Show excerpt
  [2026-03-09 19:32] xenonfun: ``` [train] Tokenizing 186,015 examples... 20,000/186,015 (4,496,870 tokens) 40,000/186,015 (8,960,555 tokens) 60,000/186,015 (13,450,804 tokens) 80,000/186,015 (17,894,743 tokens) 100,000/186,015 …
[11]1719 facts
ctx:discord/blah/watt-activation/171
- full textwatt-activation-171
  text/plain3 KBdoc:agent/watt-activation-171/e933e5f7-86d5-4705-ae35-232a38216b61
  Show excerpt
  [2026-03-09 20:43] xenonfun: ``` Mode: qa temp=0.0 top_k=40 stop=<|endoftext|> (100257) [compiled] Instruction: 'Random python example please.' ──────────────────────────────────────────────────────────── Random python example please. …
[12]23856 facts
ctx:discord/blah/watt-activation/238
- full textwatt-activation-238
  text/plain3 KBdoc:agent/watt-activation-238/a43afd2d-efa2-4c30-b5bc-f79e833271ce
  Show excerpt
  [2026-03-12 02:11] xenonfun: ``` Mode: raw temp=0.8 top_k=40 rep_penalty=1.1 stop=eos=1 [compiled] Prompt: 'The theory of quantum mechanics explains' ──────────────────────────────────────────────────────────── The theory of quantum me…
[13]33413 facts
ctx:discord/blah/watt-activation/334
- full textwatt-activation-334
  text/plain3 KBdoc:agent/watt-activation-334/51dced09-4d04-4a2d-88a6-5b07925c4a71
  Show excerpt
  [2026-03-15 12:42] xenonfun: ``` ┌───────────────────────┬────────────────┬─────────┬────────┬─────────┬────────┬───────┬───────┐ │ Config │ Attention │ loss@2K │ BPB@2K │ nits@2K │ Params │ sys_r │ sys_b │ ├──────…
[14]3958 facts
ctx:discord/blah/watt-activation/395
- full textwatt-activation-395
  text/plain3 KBdoc:agent/watt-activation-395/c3a4677b-f6e6-421f-a43d-9b98aae6ffdd
  Show excerpt
  [2026-03-19 04:10] xenonfun: On your question about "receptive field is short": Actually I misspoke. The rotor's receptive field is infinite — that's the whole point. The rotation R_t = δR_t ⊗ R_{t-1} ⊗ ... ⊗ R_0 carries information from…
[15]41515 facts
ctx:discord/blah/watt-activation/415
- full textwatt-activation-415
  text/plain2 KBdoc:agent/watt-activation-415/a43e0b02-0bd7-4b4c-92e7-09a9fd5f7cd7
  Show excerpt
  [2026-03-19 20:59] xenonfun: I don't have specific metrics for this one wired yet just letting it try see what happens while it build those (files: Screenshot_2026-03-19_at_4.58.57_PM.png) [2026-03-19 21:00] xenonfun: oh yhat is intersting …
[16]4178 facts
ctx:discord/blah/watt-activation/417
- full textwatt-activation-417
  text/plain2 KBdoc:agent/watt-activation-417/fa9398b9-69f8-43d3-bf11-35f92f05957d
  Show excerpt
  [2026-03-19 21:53] xenonfun: ``` Training: 1245 steps, bs=64, seq=8192 lr=0.01 → 0.0001, warmup=50 ──────────────────────────────────────────────────────────────────────── step 25/1245 2.0% loss=4.9797 BPB=7.184 r=0.107 SNR=-19.…
[17]4601 fact
ctx:discord/blah/watt-activation/460
- full textwatt-activation-460
  text/plain3 KBdoc:agent/watt-activation-460/b1f5e9cd-e022-4479-b238-062663ee77df
  Show excerpt
  [2026-03-21 16:56] xenonfun: ⏺ Results on fineweb_edu_bytes.bin (623MB): ``` ┌────────────────┬──────────────────────────────────────┐ │ Metric │ Value │ ├────────────────┼───────────────────────…
[18]64612 facts
ctx:discord/blah/watt-activation/646
- full textwatt-activation-646
  text/plain2 KBdoc:agent/watt-activation-646/84b51c92-144f-432e-8454-c9f4c46ef821
  Show excerpt
  [2026-04-16 03:27] xenonfun: ``` ⏺ Step 1,200: BPB 2.90 best, ~1,899 tok/s. Training well — BPB 8.6 → 2.9 in 1.2K steps. Throughput dipped slightly at step 1000 due to the DoReMi reference snapshot (cloning 50M params to disk + loading)…

Keep researching

Missing something or suspicious of what's here? Kick off a research session — a Claude agent will investigate, cite its sources, and file new facts into a dedicated context you can review before accepting into the shared view.

Training Run 1

Has Progress Stepin disputehasProgressStep

Logged Metricin disputeloggedMetric

Has Summary Metricin disputehasSummaryMetric

Has Summary Valuein disputehasSummaryValue

Inbound mentions (9)

claimsAboutSubjectClaims About Subject(1)

containsLogOutputContains Log Output(1)

expectsTweaksBeforeHfExpects Tweaks Before Hf(1)

followsFollows(1)

partOfTrainingRunPart of Training Run(1)

possessesLogsMetricsPossesses Logs Metrics(1)

presentsLogDataPresents Log Data(1)

runsOnCpuRuns on Cpu(1)

targetOfModificationTarget of Modification(1)

Other facts (274)

Timeline

References (18)

See also

Keep researching

Has Progress Stepin dispute`hasProgressStep`

Logged Metricin dispute`loggedMetric`

Has Summary Metricin dispute`hasSummaryMetric`

Has Summary Valuein dispute`hasSummaryValue`

`claimsAboutSubject`Claims About Subject(1)

`containsLogOutput`Contains Log Output(1)

`expectsTweaksBeforeHf`Expects Tweaks Before Hf(1)

`follows`Follows(1)

`partOfTrainingRun`Part of Training Run(1)

`possessesLogsMetrics`Possesses Logs Metrics(1)

`presentsLogData`Presents Log Data(1)

`runsOnCpu`Runs on Cpu(1)

`targetOfModification`Target of Modification(1)