Skip to content

SparseBrain and EnsembleBrain

Jump to bottom
Raul Montoya Cardenas edited this page Jul 29, 2026 · 2 revisions

SparseBrain and EnsembleBrain

Wiki hero

Generated with Grok Build: Grok 4.5 · xAI Imagine (/imagine)

Primary high-capacity path in src/sparse_brain.jl.

Constants

Constant Value Meaning
N 65,536 Neurons per lobe
CONN_PROB 0.01 ~1% sparse connectivity (~42M non-zeros expected before cleanup)
DT 1.0 Simulation timestep (normalized tick)
HIST_DEPTH 1000 Rolling spike-history depth
COV_SUBSAMPLE 8192 Covariance subsample size
LOBE_TAUS 10, 25, 50, 100 ms Fast / Medium / Slow / Integrator
LOBE_WEIGHTS 0.4, 0.3, 0.2, 0.1 Ensemble aggregation
V_REST / V_THRESH / V_RESET −65 / −50 / −70 mV LIF levels
SIGMA 2.0 OU noise amplitude
REFRAC_T 5 Refractory timesteps

SparseBrain Structure

SparseBrain
├── W          CuSparseMatrixCSC{Float16}   # recurrent sparse
├── W_in       CuMatrix{Float32}            # N × n_in
├── W_out      CuMatrix{Float32}            # n_out × N
├── V, S       CuVector{Float32}            # membrane, spikes
├── refrac     CuVector{Int32}
├── trace_pre, trace_post                   # STDP eligibility
├── output     CuVector{Float32}            # readout
├── n_in, n_out, tau_m
├── history    CuMatrix{Float32}            # HIST_DEPTH × N circular buffer
├── hist_idx, hist_full
├── v_thresh_dynamic
└── tick_count, total_spikes, last_spike_rate

Constructor

SparseBrain(tau_m::Float32; n_in::Int=14, n_out::Int=16, name::String="default")

Initialization steps:

  1. W (recurrent) — random COO sparse at CONN_PROB, zero diagonal (no self-connections), scale toward spectral radius ≈ 0.9 (echo-state style), upload as CuSparseMatrixCSC{Float16}.
  2. W_in — dense Xavier √(2/n_in) via host Gaussian + upload (cpu_randn_cu workaround for CUDA RNG issues).
  3. W_out — Xavier/Glorot √(2/N) so readout is non-zero from tick 1.
  4. State — V filled with V_REST; spikes, traces, refrac zeroed.
  5. History — HIST_DEPTH × N Float32 buffer on GPU (~262 MB at defaults).

Requires CUDA; validates n_in > 0 and n_out > 0.

One Timestep (step! / _step_impl!)

  1. Inhibition — clamp inhibition to [0, MAX_INHIBITION]; set v_thresh_dynamic = V_THRESH + inhib * INHIBITION_GAIN.
  2. Currents — I_rec = W * Float16.(S) (cuSPARSE); I_ext = W_in * u; OU noise σ √dt · N(0,1).
  3. Integrate — Euler–Maruyama on active (non-refractory) neurons.
  4. Spike / reset — threshold, set S, reset V, set refrac counters; update spike diagnostics.
  5. History — write S into circular buffer row.
  6. STDP traces — exponential decay + spike inject; every 10 ticks update W_out with Hebbian outer product (see STDP and Covariance).
  7. Readout — output = W_out * S.

Public step! wraps _step_impl! with Sentry capture on error.

EnsembleBrain

EnsembleBrain(; n_in::Int=14, n_out::Int=16)

Builds four named lobes with LOBE_TAUS / LOBE_NAMES, shared I/O dimensions, and prints VRAM usage after init.

Lobe Role
Fast (10 ms) Micro-structure / quick reaction; highest aggregation weight; reflex flash-learning target
Medium (25 ms) Short-term patterns
Slow (50 ms) Multi-period swings
Integrator (100 ms) Longer trend memory

Aggregation

agg_output = Σ_i weights[i] * lobes[i].output

Weights sum to 1.0. get_ensemble_output copies agg_output to CPU.

ensemble_step!

Steps every lobe on the same u and inhibition. When |reflex_signal| > 0.1, Fast lobe (index 1) uses reflex_eta * 5; others use reflex_eta. Then recompute weighted aggregate.

step!(eb::EnsembleBrain, ...) forwards to ensemble_step!.

VRAM Budget (order-of-magnitude)

Rough device footprint for one SparseBrain lobe (defaults):

Buffer Approx size
Sparse W (~42M × F16 + indices) several GB (depends on nnz)
History 1000 × 65536 × F32 ~262 MB
Dense W_out (n_out × N × F32) ~4 MB at n_out=16
Dense W_in (N × n_in × F32) ~3.7 MB at n_in=14
State vectors small vs sparse/history

Full EnsembleBrain is four lobes → plan for ≥14 GB free VRAM on an RTX-class card. Constructor logs used/total GB after init.

Diagnostics

diagnostics(brain)           # tick, spikes, rate %, V_thresh, W_out_norm
ensemble_diagnostics(eb)     # per-lobe one-liners joined by " | "

Related

Last updated: July 28, 2026 Updated by: Grok Build: Grok 4.5 Package tip reference: 4e2698c (main)

Clone this wiki locally