39

Source document

[2026-03-19 00:22] xenonfun: ## HelmholtzDynamics is the more architecturally interesting piece. Here's what it does and why it matters for your existing system. ``` What you have now: Each attention head has a single decay rate γ — one lowpass filter per head. lohe_complex_split adds a phase angle making it a bandpass, but the pole is a learned constant. Every position in the sequence sees the same filter. What Helmholtz adds — two things: 1. Multi-head resonator bank (fiber level) Instead of one γ per head, you get n_heads poles spaced at different frequencies: γ_h = |decay_h| · e^{iπh/n_heads} Head 0: φ=0 (lowpass — long memory, no oscillation) Head 1: φ=π/4 (bandpass at 1/8 of Nyquist) Head 2: φ=π/2 (bandpass at 1/4 of Nyquist) Head 3: φ=3π/4 (bandpass near Nyquist — short-range pattern detector) Each head also has its own decay rate (0.5 to 0.99), so you get a 2D grid of (memory length × frequency). The output is the mean across heads. This is essentially a learnable filterbank — like having multiple tuned radio receivers running in parallel, each listening to a different frequency band of the sequence. ``` [2026-03-19 00:22] xenonfun: ``` Your lohe_complex_split already has per-head complex poles, so the frequency part overlaps. The Helmholtz difference is that the frequencies are structured (evenly spaced to cover the band) rather than all learned independently (which can collapse to similar frequencies). 2. Base coupling (between groups) After the fiber cumsum, Helmholtz adds a second step: groups that are similar (high dot product) exchange information through a tangent-space step. The original PyTorch version uses Givens U(7) rotations for this — amplitude-preserving unitary coupling. My MLX port simplified this to a learned linear projection + tangent step on the manifold. This is different from the ring adjacency in v3. Ring adjacency couples nearest neighbors with fixed topology. Helmholtz base coupling is content-dependent — groups that happen to be aligned right now get coupled, regardless of their index. Where it fits in HarmonicMLX: The cleanest integration would be as an alternative to the Lohe sync step, not a replacement for the whole FFN. The v3 flow is: proj_in → normalize → [Lohe ring sync] → DFT readout → proj_out Helmholtz would replace the sync: proj_in → normalize → [Helmholtz fiber + base coupling] → DFT readout → proj_out The readout stays the same (DFT modes, Jacobi, eventually Clifford probes). Only the dynamics change. ```