Dontopedia

GELU

From Dontopedia, the open, paraconsistent wiki. (Last updated 2026-06-06.)

GELU has 8 facts recorded in Dontopedia across 3 references, with 1 live disagreement.

8 facts·6 predicates·3 sources·1 in dispute

Mostly:ffn forward pass sequence(1), has op count per ffn forward pass(1), defined by parameters(1)

Maturity scale raw canonical shape-checked rule-derived certified

Inbound mentions (3)

Other subjects in dontopedia point AT this entity as a value. These are inverse relationships — e.g. "X motherOf this subject" — and answer questions the forward facts can't. Grouped by predicate.

activationCompetitorActivation Competitor(1)

hasSpecialCaseHas Special Case(1)

usesActivationUses Activation(1)

Other facts (6)

The long tail: predicates that appear too rarely to warrant their own section. Filter or scroll to find a specific one. Each row links to its source.

6 facts
PredicateValueRef
Ffn Forward Pass Sequencematmul → activation → matmul[1]
Has Op Count Per Ffn Forward Pass3[1]
Defined by Parameters{"α": 1.702, "β": 0}[2]
Rdf:typeActivation Function[3]
Learnable Param Count0[3]
NoteStandard smooth[3]

Timeline

Timeline axis is valid_time — when each source says the fact was true in the world, not when Dontopedia learned about it. Retracted rows are kept for provenance; coloured stripes indicate the context kind.

8 undated
—
ffnForwardPassSequenceblah/training-and-evals/part-28
matmul → activation → matmul
—
hasOpCountPerFfnForwardPassblah/training-and-evals/part-28
3
—
labelblah/training-and-evals/20
GELU
—
definedByParametersblah/training-and-evals/20
{"α": 1.702, "β": 0}
—
typeblah/training-and-evals/23
ex:ActivationFunction
—
labelblah/training-and-evals/23
gelu
—
learnableParamCountblah/training-and-evals/23
0
—
noteblah/training-and-evals/23
Standard smooth

References (3)

3 references
  1. [1]Part 282 facts
    ctx:discord/blah/training-and-evals/part-28
  2. [2]202 facts
    ctx:discord/blah/training-and-evals/20
    • full texttraining-and-evals-20
      text/plain3 KBdoc:agent/training-and-evals-20/df884008-3d53-4aea-97bd-68748c59313f
      Show excerpt
      [2026-02-25 10:19] ajaxdavis: ``` There are a few concrete approaches, from least to most ambitious: 1. Parameterized activations (easy, high value) Instead of choosing between gelu and silu, parameterize a family that contains both a…
  3. [3]234 facts
    ctx:discord/blah/training-and-evals/23
    • full texttraining-and-evals-23
      text/plain3 KBdoc:agent/training-and-evals-23/70bec41a-b6db-4999-9455-15d7176b4205
      Show excerpt
      [2026-02-25 14:27] omega [bot]: **Symbio Search Math Problem** Current run `super_chat_20260225140824_ee4k` has a config mismatch: ``` 50000 candidates × 3000 steps × 20 gens = 3,000,000,000 total search steps But only 50,000 total trainin…

See also

Keep researching

Missing something or suspicious of what's here? Kick off a research session — a Claude agent will investigate, cite its sources, and file new facts into a dedicated context you can review before accepting into the shared view.

↑