FineWeb
From Dontopedia, the open, paraconsistent wiki. (Last updated 2026-06-06.)
FineWeb has 33 facts recorded in Dontopedia across 20 references, with 4 live disagreements.
33 facts·19 predicates·20 sources·4 in dispute
Mostly:rdf:type(5), tuned model component(3), taught skill(3)
Maturity scale
raw canonical shape-checked rule-derived certifiedInbound mentions (26)
Other subjects in dontopedia point AT this entity as a value. These are inverse relationships — e.g. "X motherOf this subject" — and answer questions the forward facts can't. Grouped by predicate.
trainedOnTrained on(3)
- Kick Model Best
ex:kick-model-best - Lohedelta a
ex:lohedelta-a - Model Fineweb E2
ex:model-fineweb-e2
testedOnDatasetTested on Dataset(2)
- Kick Model
ex:kick-model - Resonance
ex:resonance
areTrainedOnAre Trained on(1)
- Models
ex:models
comparisonDatasetComparison Dataset(1)
- Training Update 2026 03 09 14 58
ex:training-update-2026-03-09-14-58
datasetForComparisonDataset for Comparison(1)
- Wave Unified
ex:wave-unified
evaluatedOnDatasetEvaluated on Dataset(1)
- Kickmodel
ex:kickmodel
hasLessEntropyHas Less Entropy(1)
- Tinystories Distribution
ex:tinystories-distribution
:hasObject:has Object(1)
- Assertion Reading Fineweb
ex:assertion-reading-fineweb
hasPossessionHas Possession(1)
- Xenonfun
ex:xenonfun
hasTestDatasetHas Test Dataset(1)
- Run G8
ex:run-g8
hittingBpbOnHitting Bpb on(1)
- 25 Params
ex:25-params
is1-89OnIs1 89 on(1)
- Our Bpb
ex:our-bpb
onDatasetOn Dataset(1)
- Next Token Prediction
ex:next-token-prediction
performedOnPerformed on(1)
- Current Task
ex:current-task
presupposesExistsPresupposes Exists(1)
- Xenonfun
ex:xenonfun
pretrainedOnPretrained on(1)
- Model
ex:model
recitedSpecificPassageFromRecited Specific Passage From(1)
- Finite Diff G8 100k
ex:finite-diff-g8-100k
referencesFinewebDatasetReferences Fineweb Dataset(1)
- Text
ex:text
refersToDatasetRefers to Dataset(1)
- Fineweb Only
ex:fineweb-only
switchToSwitch to(1)
- Plan
ex:plan
targetsDatasetTargets Dataset(1)
- Planned Fresh Run
ex:planned-fresh-run
trainedOnDatasetTrained on Dataset(1)
- Lohe Delta a
ex:lohe-delta-a
usedInRetokenizationUsed in Retokenization(1)
- Eot Tokens
ex:eot-tokens
Other facts (27)
The long tail: predicates that appear too rarely to warrant their own section. Filter or scroll to find a specific one. Each row links to its source.
27 facts
| Predicate | Value | Ref |
|---|---|---|
| Rdf:type | Dataset | [13] |
| Rdf:type | Training Phase | [14] |
| Rdf:type | Dataset | [15] |
| Rdf:type | Dataset | [16] |
| Rdf:type | Dataset | [17] |
| Tuned Model Component | Embeddings | [14] |
| Tuned Model Component | Attention Patterns | [14] |
| Tuned Model Component | Positional Encoding | [14] |
| Taught Skill | academic text | [14] |
| Taught Skill | varied topics | [14] |
| Taught Skill | complex syntax | [14] |
| Presupposes Prior Training | Known to Reader | [1] |
| Contextual Prior Pretraining | Previous Epoch | [1] |
| References | Dataset | [2] |
| Influences Generation | {} | [3] |
| Is Referenced As | Dataset | [4] |
| Text Dataset Style | null | [5] |
| Has Similar Patterns | Shakespeare | [6] |
| Serves As Dataset | True | [7] |
| Used for Bpb | null | [8] |
| Evaluation Dataset | null | [8] |
| Contains Passage About | Florida Museum of Natural History | [9] |
| Available But Unmentioned | null | [10] |
| Standard Dataset | Bpb Testing | [11] |
| Precedes Sequentially | Tinystories | [12] |
| Training Effect | general language structure | [14] |
| Final Val Loss | 6.03 | [14] |
Timeline
Timeline axis is valid_time — when each source says the fact was true in the world, not when Dontopedia learned about it. Retracted rows are kept for provenance; coloured stripes indicate the context kind.
—
presupposesPriorTrainingblah/watt-activation/part-143
ex:known-to-reader
—
contextualPriorPretrainingblah/watt-activation/part-143
ex:previous-epoch
—
referencesblah/watt-activation/part-164
ex:dataset
—
influencesGenerationblah/watt-activation/part-169
{}
—
isReferencedAsblah/watt-activation/part-228
ex:dataset
—
textDatasetStyleblah/watt-activation/part-244
null
—
hasSimilarPatternsblah/watt-activation/part-348
ex:shakespeare
—
servesAsDatasetblah/watt-activation/part-465
ex:true
—
usedForBpbblah/watt-activation/part-472
null
—
evaluationDatasetblah/watt-activation/part-472
null
—
containsPassageAboutblah/watt-activation/part-474
ex:florida-museum-of-natural-history
—
availableButUnmentionedblah/watt-activation/part-648
null
—
standardDatasetblah/watt-activation/part-696
ex:bpb-testing
—
precedesSequentiallyblah/watt-activation/part-145
ex:tinystories
—
typeblah/general/122
ex:Dataset
—
labelblah/general/122
FineWeb
—
typeblah/watt-activation/144
ex:TrainingPhase
—
labelblah/watt-activation/144
FineWeb
—
trainingEffectblah/watt-activation/144
general language structure
—
tunedModelComponentblah/watt-activation/144
ex:embeddings
—
tunedModelComponentblah/watt-activation/144
ex:attention-patterns
—
tunedModelComponentblah/watt-activation/144
ex:positional-encoding
—
finalValLossblah/watt-activation/144
6.03
—
taughtSkillblah/watt-activation/144
academic text
—
taughtSkillblah/watt-activation/144
varied topics
—
taughtSkillblah/watt-activation/144
complex syntax
—
typeblah/watt-activation/281
ex:Dataset
—
typeblah/watt-activation/472
ex:Dataset
—
labelblah/watt-activation/470
fineweb
—
typeblah/watt-activation/470
ex:Dataset
—
labelblah/watt-activation/645
fineweb
—
labelblah/watt-activation/679
FineWeb
—
labelblah/watt-activation/693
fineweb
References (20)
20 references
ctx:discord/blah/watt-activation/part-143ctx:discord/blah/watt-activation/part-164ctx:discord/blah/watt-activation/part-169ctx:discord/blah/watt-activation/part-228ctx:discord/blah/watt-activation/part-244ctx:discord/blah/watt-activation/part-348ctx:discord/blah/watt-activation/part-465ctx:discord/blah/watt-activation/part-472ctx:discord/blah/watt-activation/part-474ctx:discord/blah/watt-activation/part-648ctx:discord/blah/watt-activation/part-696ctx:discord/blah/watt-activation/part-145ctx:discord/blah/general/122- full textgeneral-122text/plain3 KB
doc:agent/general-122/f35e5716-09b5-4b03-a9a3-6a69c152c03eShow excerpt
[2026-03-25 23:41] traves_theberge: AI-powered code review that runs anywhere - your terminal, your CI pipeline, or inside your AI coding agent. Overview OpenLens is an open-source code review tool that runs multiple specialized AI agents …
ctx:discord/blah/watt-activation/144- full textwatt-activation-144text/plain3 KB
doc:agent/watt-activation-144/3be4aaf8-37ca-4d9d-bc3c-e22f23534527Show excerpt
[2026-03-09 15:00] xenonfun: seems to really like it: step 100/64663 0.2% loss=5.3857 ppl= 218.3 lr=1.50e-05 668ms 12,265tok/s eta=719min step 200/64663 0.3% loss=4.7992 ppl= 121.4 lr=3.00e-05 667ms 12,277tok/s eta=…
ctx:discord/blah/watt-activation/281- full textwatt-activation-281text/plain2 KB
doc:agent/watt-activation-281/f91e8b96-d755-417a-ba44-47aabb5f5db2Show excerpt
[2026-03-13 23:22] xenonfun: ``` ⏺ This is the result we needed to see. ┌───────┬──────────────────────┬─────────────┬────────────┬───────┬─────────┐ │ Codes │ Positions/code/batch │ Utilization │ Perplexity │ Top-1 │ Entropy │ ├────…
ctx:discord/blah/watt-activation/472- full textwatt-activation-472text/plain2 KB
doc:agent/watt-activation-472/55d07941-f15c-4291-87cc-5e7dd6847ff0Show excerpt
[2026-03-21 19:37] xenonfun: it already told me the first pass it did in 15m a few hours ago was gonna take 4 weeks. (files: Screenshot_2026-03-21_at_3.37.10_PM.png) [2026-03-21 19:42] xenonfun: ⏺ Finite-diff done: BPB 2.04 in 29.3 min. Ana…
ctx:discord/blah/watt-activation/470- full textwatt-activation-470text/plain3 KB
doc:agent/watt-activation-470/ef3b30df-5bf6-491e-86c9-9618c45736fcShow excerpt
[2026-03-21 19:00] xenonfun: ``` ⏺ g8 finished. BPB 2.04 with 25 params. Final multi-group results: ┌────────┬────────┬─────────────┬──────────┬───────┬───────┐ │ Groups │ Params │ Param bytes │ Best BPB │ tok/s │ Time │ ├───────…
ctx:discord/blah/watt-activation/645- full textwatt-activation-645text/plain3 KB
doc:agent/watt-activation-645/5c424827-0967-4bac-bbcc-cb1b4a153e35Show excerpt
[2026-04-16 02:39] lisamegawatts: oh nmd [2026-04-16 02:39] lisamegawatts: its all good [2026-04-16 02:40] lisamegawatts: i thought it still had the books stuff but it doesn't and you can filter it, so that should be plenty for 50m chinchil…
ctx:discord/blah/watt-activation/679- full textwatt-activation-679text/plain2 KB
doc:agent/watt-activation-679/e7b96ce5-bed6-4b84-baf1-a773f080c0dbShow excerpt
[2026-04-22 14:42] xenonfun: ``` ⏺ Pulling the per-step bpb trajectory from metrics.jsonl of the two most-comparable prior runs: step | unified_rotor_gpu_25m | lohe_delta_25m_replication | lohe_delta_16L_G16_H4_R16_doremi …
ctx:discord/blah/watt-activation/693- full textwatt-activation-693text/plain3 KB
doc:agent/watt-activation-693/8372b985-fcfe-4f68-a730-4cd9d14a0fb9Show excerpt
[2026-04-28 23:22] xenonfun: KickModel (sparse extreme — top pick for fast iteration) 25 params total at g=8 (yes, twenty-five). BPB 2.04 on fineweb. Architecture: ∂ₜΨ = [Ω, Ψ] − γ⟨Ψ⟩ + kick, G independent S³ oscillators (3G+1 params), exp…
See also
Keep researching
Missing something or suspicious of what's here? Kick off a research session — a Claude agent will investigate, cite its sources, and file new facts into a dedicated context you can review before accepting into the shared view.