Skip to content

NeuroBrix 0.6.1

Choose a tag to compare

@benkelaya benkelaya released this 07 Nov 12:15
· 192 commits to main since this release

A first run never ends in a Traceback, the package names none of the project's machines, and the documentation says
where NeuroBrix runs and how to start. The kernels that compute a model are 0.6.0's: its final gate is carried, and five
reference models through the entry point give the same outputs; what changed is how a new machine starts (its first-run
measures, its arena factor, its home), what the packages carry, and Windows.

The Apple certified directory is proven again under the published Triton build for Apple GPUs (18 352 kernel settings);
a few Apple kernel settings for some models are certified in 0.6.2; until then they are tuned once at first run and kept.

Changed

  • The README and the documentation say where it runs and how to start: Linux, Windows and macOS; on a Mac,
    NeuroBrix's Triton build for Apple GPUs (NeuroBrix/triton-ext); every neurobrix import example carries
    --accept-license, which the import asks of a model with a license (the README's first run stopped there).

Fixed

  • A first run on a new machine never ends in a Traceback because its arena probe ran out of memory: the first launch
    measures the Triton arena's hold on the cache's smallest model by doubling its request while a prediction says the
    next fits; on an Apple GPU the prediction let a request through that the device could not hold, the probe's process
    ran out of memory, and its error came back as the interpreter's shutdown noise ("UNEXPECTED ERROR" at the first
    neurobrix run). The device's refusal is now the measure's bound (the largest request that fitted is kept), and the
    probe's own error is said by name.
  • A new machine's first run does not measure the Triton arena: the first launch probed what the Triton arena holds
    over a component's activations on the cache's smallest model, and that probe scaled its inputs unlike the request
    Prism prices: on a fresh Mac it measured 28 where the machine's value is about 1.5, so Prism reserved some 19 times the
    activation memory, and once corrected it still depended on the model imported (2.4 on a VAE) and swept uncertified
    kernels for 40 minutes. The factor is now declared — by the device's vendor file, else by its family (Apple 1.477,
    measured on an M4 Pro; NVIDIA 1.35, measured on Volta; AMD and Intel the NVIDIA value until measured) — and a first
    run never probes it; a value an earlier first launch wrote into a profile is no longer read. The probe scales only what
    the priced request scales (a tool, until it is bounded to certified shapes).
  • A run starts on Windows: the plan's resident term is read from the process's working set (GetProcessMemoryInfo)
    where it needed /proc or libproc, and a first-launch read-rate measure Windows cannot make (no way to read past its
    page cache) is declared unmeasured and the run reads serially, said on its launch line. Written and tested with Windows
    simulated: the project has no Windows machine yet.
  • Text in UTF-8 on every platform: a generated text written to a file, the vendor tokenizer files read at load, the
    agent's file edits and transcript, and the console output now use UTF-8 where the platform's default code page is
    another (Windows' cp1252: a wrong vocabulary read silently, or a run ended by UnicodeEncodeError); a debug dump's file
    name drops every character a file name refuses on some system; a stale install lock is asked of the kernel on Windows
    instead of signalling the process (signal 0 is Ctrl+C there).
  • The engine depends on nbxtensor 0.0.5: a certified setting the library does not serve says why (another code
    generator, or another memory class).
  • A configured home is followed everywhere: with NEUROBRIX_HOME set, the replay cache, the license acceptances,
    the hardware profiles, the calibration records, the render checkpoints and the serving daemon's files still went to
    ~/.neurobrix; they now follow the configured home.
  • neurobrix doctor on a Mac without PyTorch told the user to pick a CUDA build of PyTorch; it now says
    pip install torch, PyTorch's own macOS wheel.
  • The package names none of the project's machines: the snapshots root it shipped was a path of the project's own
    machine (wrong on every other machine) and is now ~/hf_snapshots
    (NEUROBRIX_HF_SNAPSHOTS overrides it); the census input records name the files they read relative to the engine's
    home (home:) instead of a user's home directory; the certified directory's proofs no longer record the host name
    of the machine that certified them (85 723 removed, and the certifier no longer writes it).
  • A first run on a busy storage is never a Traceback: at its first launch the engine measures how many parallel reads
    the machine's model cache serves best; where no stable read rate was found (a storage another reader was using — a
    clean container whose cache was a network mount), the run ended in "UNEXPECTED ERROR: ZERO FALLBACK: no stable
    read-rate measure" and a Traceback. The run now reads serially, its launch line says so ("io workers: 1 for this
    run — ...; measured again at the next run"), and the profile keeps no count, so the next run measures again.

Install: pip install neurobrix — the Triton engine, no PyTorch. With the PyTorch engine as well: the PyTorch build for
your hardware, then pip install "neurobrix[pytorch]". On an Apple-silicon Mac (macOS 15 or later, Python 3.12):
pip install neurobrix, then python -m nbxtensor install-metal. neurobrix doctor says what each engine needs on your machine.