16

Source document

[2026-02-25 16:47] omega [bot]: **Why `relu×gelu` gen-1 only connects to `relu` gen-0 (not `gelu` gen-0)** In composed activation evolution, each candidate has **one parent**. The lineage tree is a proper tree, not a DAG. When `relu` gen-0 gets an `inject_gate` mutation, it becomes `relu×gelu`: ``` relu (gen-0) → inject_gate mutation → relu×gelu (gen-1) ``` The `gelu` in the formula was **injected by the mutation operator** — it randomly picked `gelu` from the basis pool (`silu`, `relu`, `gelu`, `identity`, `square`) as the gating function. It did NOT come from the `gelu` gen-0 candidate. The `gelu` gen-0 candidate is a completely separate organism with its own lineage tree. The same basis op can appear in multiple unrelated candidates — `gelu` the basis atom is just raw material that any mutation can grab from the pool. **Analogy:** If two animals both have eyes, that doesn't mean one descended from the other. Eyes (like `gelu`) are a building block that evolution can independently discover and combine. The lineage tree tracks actual parent-child reproduction, not shared components. [2026-02-26 21:34] optionalsecurity: https://gemini.google.com/image-gen [2026-02-26 21:34] optionalsecurity: Nano banana 2 [2026-03-02 22:15] ajaxdavis: https://www.ntik.me/posts/voice-agent [2026-03-21 21:14] xenonfun: https://github.com/openai/parameter-golf [2026-03-21 21:16] xenonfun: https://openai.com/index/parameter-golf/#credit-form [2026-03-22 02:29] traves_theberge: That’s wild [2026-03-22 02:30] traves_theberge: I love that [2026-03-22 02:30] traves_theberge: Bring on the SLMs [2026-03-23 20:27] xenonfun: https://x.com/i/status/2031845139526652236 [2026-03-26 04:44] traves_theberge: https://github.com/Vaibhavs10/insanely-fast-whisper [2026-03-27 10:02] lisamegawatts: https://x.com/aiatmeta/status/2037153756346016207?s=46&t=OS71yaTltL7EclsAy81ZcQ [2026-03-27 10:03] lisamegawatts: https://x.com/huggingmodels/status/2037384376196170212?s=46&t=OS71yaTltL7EclsAy81ZcQ [2026-04-06 17:02] lisamegawatts: i was reading through the claude emotions paper, and i wonder if the ssh mail scenario was similar enough to their testing that it prompted the research into sycophancy petition https://transformer-circuits.pub/2026/emotions/index.html [2026-04-07 13:03] ajaxdavis: i quote tweeted ths the other day, weird key word matching which seems crazy, seems like from that url they dont do that everywhere obviously (files: Screenshot202026-04-0720at2011.02.29E280AFPM.png) [2026-04-07 13:03] ajaxdavis: ファックユー [2026-04-07 17:48] ajaxdavis: also... (files: 48c0a196-33f6-41bc-a43c-8ea945d365b9.png) [2026-04-07 18:35] ajaxdavis: lol kicked off new project, building a fastq dna parser (and other packages) in typescript with spir-5 custom instructions aha (probably just gonna do sample human siblings and calculating centimorgans to begin with) [2026-04-07 18:37] traves_theberge: wild man [2026-04-23 06:22] lisamegawatts: https://github.com/galilai-group/lejepa