10

Source document

[2026-03-20 11:25] foxhop.: awesome new video card with 12G & over 3k cuda cores! [2026-03-20 11:25] foxhop.: ? [2026-03-20 11:27] foxhop.: "We're building the disk." [2026-03-20 11:28] foxhop.: this screams GPT switch all "the" toward "a" [2026-03-20 11:28] foxhop.: I give that suggestion when working with the bots [2026-03-20 11:30] foxhop.: > entire GPU layer is one 130-line Nim file plus a 280-line CUDA kernel file. Half the code. No hacks. [2026-03-20 11:30] foxhop.: so cool. [2026-03-20 11:35] foxhop.: > When we swap the RTX 3060 for a Tenstorrent Blackhole, the change is one file — replace cuBLAS calls with TT-NN calls. [2026-03-20 11:35] foxhop.: 🔥 ⚽ [2026-03-24 04:52] rolandnsharp7643: https://github.com/rolandnsharp/nimllm [2026-03-24 04:56] rolandnsharp7643: hehe claude getting caught up in the moment . Nim ended up being way faster than the ocaml code when we converted over to GPUs because of no garbage collection or something [2026-04-01 02:51] rolandnsharp7643: https://www.youtube.com/watch?v=l-OLgbdZ3kk&t=11s [2026-04-01 08:23] rolandnsharp7643: New Direction — oscillator neural network in OCaml, massive parallel CPU architecture instead of GPU. No matrix multiplication math so pyTourch can't even help us here. Yeah, back to OCaml now that we aren't using GPUs. we need high performance functional composition of oscillators and OCaml wins. A neural network where the core computation is FFT convolution through a bank of damped harmonic oscillators — one equation: H(ω) = 1/(ω₀² - ω² + 2iγω). Each byte of text strikes the oscillator bank like a hammer hitting a bell. The ringing IS the representation. No attention mechanism, no transformer blocks. The whole thing is in OCaml, designed from the ground up for parallel CPU execution. Currently training on Shakespeare using OCaml 5 Domains across 12 cores with batch-parallel gradient computation. Current results: - 2.37 BPC on Shakespeare (down from 8.0 random baseline) - Generating proto-English: "is my t e oue pe ge bid avose aryse sint mowemis in" - Real words emerging: "war", "The", "his", "are", "port", "by" - Just added SineGate MLP (x × sin(x) activation — oscillation as nonlinearity) The vision: This architecture doesn't need GPUs because there's no giant matrix multiply bottleneck. The work is thousands of small independent FFT convolutions — one per oscillator. Each oscillator can run on its own core. The scaling path is more cheap CPU cores, not bigger GPUs. Eventually: RISC-V microcontrollers at $2 each, one oscillator per chip, communicating predictions and errors over SPI. A $200 cluster instead of a $30,000 GPU. The encoding is pure physics — causal convolution with damped oscillator impulse responses via FFT. The processing uses a learned transform with SineGate activation. The readout is weight-tied — the same drive signatures used to strike the bank are used to listen for the prediction. Strike and listen are the same resonance.