Skip to content

Releases

Apsi 2.6.2

Choose a tag to compare

@github-actions github-actions released this 21 Nov 16:43

2.6.2 — large graph and language models on one card, training that resumes exactly and learns from its own output, half the optimizer's memory, your own GPU kernels (21 November 2026)

Install: apsi.dev/download — ApsiSetup-2.6.2-x64.exe (Windows 10/11),
ApsiSetup-2.6.2-linux.sh (any Linux, WSL), apsi_2.6.2_all.deb (Ubuntu / Debian), or from this source:
pip install numpy, then ./apsi.sh / apsi.bat. What's new, with examples: docs/WHATS-NEW.md.

Larger models fit on one card: activations recomputed only where they do not fit (planned again when a batch grows),
optimizer moments in bfloat16, more graph layers as card kernels, and batches made from the model's own output while
it trains. Everyday code runs as fast as 2.6.0 (the same programs on one GitHub runner:
every row within run-to-run noise; apsi check 4 % faster). Tested: 507 toolchain tests and 149 Apsi tests on every push (Linux, macOS,
Windows; Python 3.12–3.14). New example: examples/47_train_your_llm.apsi.

Training that resumes exactly, and batches made while it trains

  • Exact resume of everything: a checkpoint now keeps the random generators, the step counts of parameter groups
    that step every batch, and your program's own state: fit_batches(..., state=record(get=fn() => text, set=fn(text) => ...)) (a replay buffer, running averages, a curriculum position). Five steps + resume + five steps
    give the same loss as ten in one go (relative difference below 1e-10, on the card and on the processor). An epoch
    that has nothing left after resuming is no longer reported as a loss of 0.
  • Batches made when training reaches them: an item record(later=fn, end_step=false) in the batch list is
    called at its turn and its batches are trained there; inside it training_params() reads the weights of that
    moment and training_update(params, only=["part.sub"]) writes some back (a part trained by an optimizer of your
    own). Reinforcement learning from the model's own samples, data that depends on the model, curricula.
  • Steps of their own length: accumulate=0 ends a step at the batch whose targets say end_step=true; each
    batch divides its gradient by its step_items. Parameter groups with every="batch" take their own AdamW step
    after every batch (hold=true keeps the gradient for the next one): a decoder with its own optimizer inside a
    larger model.
  • card_loss(..., scale="field") (a batch that counts more, less or with a negative sign) and labels none (the
    whole loss in rest, which may then read the logits).

Less memory, the same updates

  • adam(state="bf16") / adamw(state="bf16"): the optimizer's two moments in bfloat16 with stochastic rounding
    (random low bits added before cutting, so the rounding averages out): 4 bytes per weight less on the card, the
    update computed in float32 as before. Checkpoints keep the 16-bit moments; the processor path rounds the same way.

The language

  • Raw strings r#"..."#: no escapes, several lines, end at " followed by the same number of #
    (r##"..."## when the text holds "#). Regular expressions, Windows paths, other languages' source.
  • pymodule(name, source): Python source kept inside your program becomes a module (--allow-python), so a
    program ships as .apsi files only. Its code is the program's own: its _private names are reachable, a
    library's are not (--allow-python=all opens those too).
  • save_safetensors keeps 0-d tensors as scalars (shape [], as other programs write them).
  • New example: examples/47_train_your_llm.apsi — a tokenizer, a LLaMA / Qwen3-style decoder, training and text
    written by the model, in 40 lines.

Models that did not fit now train

  • Activation recompute (fit_batches(recompute="auto"), the default on a card): the first step measures the
    memory; only the blocks whose activations do not fit are computed again in backward (residual / transformer
    blocks, hgt, global attention, graph transformer layers). checkpoint(layer) always recomputes. Same weights to
    the last bit as keeping every activation.
  • Global attention in pieces of whole rows (APSI_GA_PIECE): nothing per edge is kept between forward and
    backward, so memory grows with the nodes, not nodes × keys.
  • Weights reach the card in pieces; every parameter starts on a 32-byte boundary, so one odd-sized head no longer
    pushes the matrices after it off the fast 16-bit products.

Your own kernels, your own losses

  • gpu_layer(forward, backward, init, out): your @gpu kernels as a layer of a model the GPU trainer trains,
    on the trainer's own buffers.
  • card_loss(output, rest, labels, weights=): a model with several outputs; the language model's cross-entropy
    stays on the card, the other outputs go to rest on the processor. weights= gives each label its own weight
    (sum(w ce) / sum(w)), exactly on the card.
  • Parameter groups: fit_batches(groups=[record(params=[...], lr=, b1=, b2=, eps=, weight_decay=, clip=, decay="matrices"|"all"|"none"|[parts], no_decay=[...]), ...]) — each part of a model with its own AdamW settings
    and its own gradient clipping, scaled by the schedule; the same update on the card and on the processor.
  • grad_scale(k): a chosen share of the gradient flows back (0 = stop, 0.08 = a soft bridge).

Graphs and layers

  • pick_rows with nothing picked gives zeros as on NumPy. crypto.crc32, hash_int.
  • Stateful layers replay their running statistics in backward, and model_state / set_model_state save them;
    fit_batches checkpoints carry them.

Model files of other programs

  • models.named_tensors(params, rows, shapes) / from_named_tensors(like, named, rows): a parameter tree under
    another program's tensor names (fused weights, transposes, scalars), so either program reads the other's
    checkpoints; load_named_tensors(path) reads a .safetensors file under its exact names and
    safetensors_metadata(path) its text metadata (a tokenizer, a configuration).
  • GPU checkpoints record where each weight sits and refuse a checkpoint of another layout.

Servers

  • app.group(prefix): routes under a path prefix (let api = app.group("/api/v1"), @api.get("/users/{id:int}"));
    groups nest, and take WebSockets too.
  • app.compress(min_size=1024, level=5): gzip for clients that send Accept-Encoding: gzip — text, HTML, JSON,
    JavaScript, CSS, XML and SVG bodies of at least min_size bytes; images, small bodies, HEAD, 304 and responses
    that set their own Content-Encoding are sent as they are; Vary: Accept-Encoding is added. app.handle() gives
    tests the decoded body.
  • app.ready(): GET /ready answers 200 while serving and 503 (Retry-After: 1) once app.stop() was called,
    so a load balancer or orchestrator drains the instance; app.health() now also reports the uptime.

Fixes

  • recompute="auto" plans again when a batch larger than any planned for arrives (a batch made while training,
    record(later=), or a larger graph): the blocks that no longer fit are recomputed from then on and their buffers
    freed, instead of the card filling up. The results stay the same to the last bit.
  • The bfloat16 copy of a long array (the whole weight array of a large model) is made in launches the kernels can
    cover; before, the tail stayed stale and the 16-bit products were refused by the self-check.
  • layers.fit_batches passes exact_sums= to the GPU trainer; predict on the card takes several inputs,
    several outputs and the batch's graph.

Apsi 2.6.0 (Improved version)

Choose a tag to compare

@github-actions github-actions released this 20 Nov 07:00

2.6.0 (Improved version) — exact training on any number of cards, heterogeneous graphs, training that finishes, fewer tokens (20 November 2026)

Install: apsi.dev/download — ApsiSetup-2.6.0-x64.exe (Windows 10/11),
ApsiSetup-2.6.0-linux.sh (any Linux, WSL), apsi_2.6.0_all.deb (Ubuntu / Debian), or from this source:
pip install numpy, then ./apsi.sh / apsi.bat. What's new, with examples: docs/WHATS-NEW.md;
everyday code runs 1.22× faster than in 2.4.4 (geometric mean); the training guide: docs/TRAINING.md.

Faster than PyTorch on the same card: a 127 M GPT trained for 2 epochs on an RTX 3060 at 20,708 tokens/s against
PyTorch 2.14 bf16's 18,809 at the same temperature (1.10×), with Apsi's own 16-bit kernels (no cuBLAS), the card never
above 79 °C.

Tested: 445 tests on every push (Linux x64 / ARM64, macOS Apple silicon / Intel, Windows; Python 3.12–3.14) and 112
on a real NVIDIA RTX 3060 (13 of 13 kernel checks against the exact reference; heterogeneous graph models and a transformer
trained twice with identical weights; steps and resume bit for bit; the NaN guard).

Setup 1.50.0: the GPU helper step takes seconds (it no longer measures the card during setup), the wizard shows each
stage and the elapsed time, and uninstalling removes the GPU helper first (apsi gpu remove as administrator does it
by hand).

Binary layouts and embedded work

  • use layout: c_struct (natural alignment or pack=, little / big endian, arrays, nested structs, bit
    fields), pack / unpack / view (fields read and written in place), numpy_dtype, and c_header() with
    static assertions of every offset; register_map for device registers (a C header of volatile accesses, read and
    write over a memory image). Offsets checked against the C compiler; the headers compile with GCC and Clang.
  • FFI: ptr_void / ptr_u8 take a bytearray, a tensor or a layout view (C writes into Apsi's memory);
    u8 u16 u32 u64 i8 i16 arguments.
  • apsi convert takes Jupyter notebooks (.ipynb, also inside folders): code cells in order, markdown as
    comments, % / ! lines kept as marked comments.
  • enum declarations (enum Color: RED, GREEN = 5, BLUE, or one member per line) and match documented,
    with built-in types in class patterns (case str() as s:, case int(n) if n > 100:).
  • Base-16, 2 and 8 integers: 0xFF, 0b1010_0101, 0o755. docs/EMBEDDED.md.

Sealed model packages

  • pack_model / load_package / package_info (use layers): a trained model's architecture and weights in
    one encrypted file (.apsipkg: ChaCha20 + HMAC-SHA256, a password through scrypt or a 256-bit key); nothing in it
    can be read without the key, a changed byte is refused, and the architecture is data, never code.
    apsi package lib MODEL NAME builds a library around it (load, predict, info) that needs only Apsi, not the
    program that trained the model. Every layer now remembers the arguments it was made with.

Graph batches that never make the card wait

  • Graphs of any size in one batch: batch_graphs([(x, graph, y), …]) (nodes renumbered; edges, relations,
    node types and edge features kept), graph(…, edge_features=) and gine (GIN with edge features). Same training
    on NumPy and the GPU trainer.
  • The next batch's graph is prepared on another core while the card trains on this one (sorted edges, offsets,
    pooling order: the forms the layers asked for last time), and its buffers on the card come from a pool reused
    from batch to batch instead of being allocated and freed (each free waited for the card). Graphs of any size and
    any degrees per batch, the same training. APSI_GRAPH_PREFETCH=0 turns the thread off.

Faster everyday code

  • Facts the compiler can prove are no longer checked at run time: a typed Int/Float parameter that is never
    rebound is a plain number for the whole call (a comparison of two such numbers is a plain condition; returning it
    needs no second check); for i in 0..n with whole-number ends runs on Python's own range and i is a known
    integer (indexes and f-strings use it directly); a name bound once to a list, dict, set or text literal calls its
    methods, iterates and stores directly. Same results, same errors.
  • Structs are built directly: a struct whose fields are untyped, Int or Float gets a constructor of its own
    (no general plan per call): creating 200,000 structs is 1.8× faster.
  • Measured against 2.4.4 on the same machine (GitHub runner): recursive calls (fib 25) 1.5×
    faster, a word count 2.1×, 200k f-strings 2.3×, 200k structs 1.7× (GitHub runner, geometric mean of the 12
    measurements 1.22×); the eight heavy @native programs are unchanged (the native code generator was already at
    C's speed).
  • Products whose sizes are not multiples of 4 (a SwiGLU width of 2730, an odd vocabulary) run on the tiled
    tensor-core kernels through zero-padded copies instead of the general kernel (APSI_GEMM_PAD=0 turns it off).
  • bfloat16 training no longer keeps compensated weights by default: the compensation (4 bytes per parameter and
    a fifth array for AdamW to read and write every step) stays on by default for fp32 training, where it keeps
    updates smaller than float32's step; in bf16 the products round every gradient far more coarsely, so it only cost
    memory and time: 0.5 GB less on the card for the 127 M GPT, and a lighter optimizer step. compensated=true or
    APSI_GPU_COMPENSATED=1 turns it on.
  • Fixed power, the fans hold the temperature: while training, the card runs at a fixed 145 W
    (APSI_GPU_TRAIN_POWER=140W or 85%) and never below 135 W (APSI_GPU_POWER_FLOOR; never above the card's
    default), and its fans are steered from 70 % up to 100 % (APSI_GPU_FAN_MIN / APSI_GPU_FAN_MAX) to hold
    78.5 °C: the error now, the error 4 s ahead along the slope, and a feed-forward from the power drawn (heat that is
    in the die but not yet on the sensor). Only with the fans at full and the card still at the ceiling does the power
    come down, slowly and no further than 135 W; past that the card runs warmer rather than slower (pauses only as
    protection from 87 °C). Before, the power limit itself held the temperature: the clocks fell with it (to 106-112 W
    on an RTX 3060), and at its floor the work was paused. The driver's fan curve comes back when the card is well
    below the ceiling and at exit; APSI_GPU_FAN=0 leaves the fans alone.
  • bfloat16 is the default precision, said at the start of every run (precision="fp32" or APSI_PRECISION=fp32
    for float32-exact products as three bfloat16 products, 3× the time; the low-bit formats as before).
  • Each card measured once, at install: apsi gpu setup (the installer's GPU step, and the GPU Tuner) now also
    runs apsi gpu calibrate: about 2.5 minutes of matrix products at a trial power with the fans at the top and then
    the bottom of their range; each temperature curve is fitted to a first-order model, so the settled temperature is
    known without waiting for it. From the degrees per watt at each fan speed come the highest power the fans hold at
    77.5 °C (never below the card's own limit, never above its default) and the quietest fan range that holds it,
    saved in ~/.apsi/gpu-profile.json and used by every run. Without a measurement: 85 % of the default power, fans
    70–82 %. Safety: the trial power drops 10 % at the ceiling + 1, the load stops at the ceiling + 3, and the card's
    own fan curve and power limit come back at the end.
  • Fans on GeForce cards under Windows: where NVML refuses to set fan speeds, Apsi uses the driver's NvAPI fan
    control (the interface the vendors' tuning tools use); a probe at the start says which, and the training report's
    control line says what holds the temperature. The calibration restarts a phase whose trial power had to come
    down (one constant power per fitted curve) and measures on the driver's fan curve when the fans cannot be set.
  • The bfloat16 hidden layer's products on the fastest builds: three of its four products (no activation on the
    way in) may now run on the cp.async builds, reading the hidden layer and the weights' bfloat16 copies directly;
    every faster build the card picks is first checked against the base build on the same inputs and never used if
    it differs (APSI_GEMM_VERIFY=0 skips the check).
  • Fewer stalls from the power limit: when it holds the temperature, changes are at least 1 W and 1 s apart
    (each change makes the driver re-plan the clocks).
  • APSI_GPU_TEMP=max: the card as PyTorch runs it: the full default power limit for the run (170 W on an RTX
    3060, put back at exit), the card's own fan curve and temperature limits (the driver slows the clocks near 83 °C);
    Apsi only reports. For the fastest training and for comparisons with programs that hold nothing.
  • Fixed: a linear layer could overwrite the gradient it was reading. The gradient buffers are shared by
    nodes two apart, and a linear layer with a fused activation made the layer two places further (linear, act,
    linear) one of them, so dy W^T wrote over its own dy; with the weight gradient on the second stream at the same
    time, the RTX 3060 check of three models (cnn, graph transformer, heterogeneous graphs) differed. Such a layer now writes to a buffer
    of its own (tests/test_gpu_train.py: no product writes over its input); with the fix the check agrees with the
    second stream on. The bfloat16 MLP hidden layer (APSI_BF16_HIDDEN=1) is opt-in: with it, the fused-attention
    transformer of the check did not give the same weights twice.
  • Two more product builds the card times: slices of 32 in two stages (40 KB of tiles), so two blocks share a
    multiprocessor; on th...
Read more

Apsi 2.4.4

Choose a tag to compare

@github-actions github-actions released this 21 Nov 18:31

2.4.4 — Python, exactly; real time; data, databases and the web

AI training on your own GPU — faster than PyTorch, nothing else to install

  • fit(…, backend="gpu") trains with kernels Apsi generates itself from the NVIDIA display driver alone: tiled
    tensor-core matrix products with the bias, activation and residual fused in, fused attention, one-kernel AdamW, the
    data set kept on the card. Measured on an RTX 3060, same model and same data: 151,744 tokens/s against PyTorch
    2.14 bfloat16's 126,993 (1.19×)
    , every epoch at the same speed (153,552 · 154,009 · 151,667), the card held near
    77 °C. fp8 144,176, fp4 142,284 tokens/s. A 127 M-parameter GPT: 18,074 tokens/s held at 77 °C.
  • Four builds of every matrix product, timed by the card on the real shapes the first time; the fastest is kept.
  • Several cards: device="cuda:all" — reduce-scatter, sharded optimizer state, all-gather; identical results
    to one card with the whole batch.
  • Steady temperature without pauses: the card's power limit is moved to hold 77 °C. Set it up once with the
    Apsi GPU Tuner (download page: Apply / Restore, Windows and Linux) or apsi gpu setup; apsi gpu hold 77
    holds the temperature for any program.
  • ema= and ewc= for smoother weights and continual learning; low-bit training in fp8, fp6, fp4, int8, int4.
  • apsi serve-model: the OpenAI API with streaming, a fair queue and metrics; 0.5B, 3B and 14B models fully on
    a 12 GB card.

Python conversion: exact, complete

  • Libraries come along: the installed libraries a program imports that are written in Python (requests,
    urllib3, idna, certifi …) are converted with it into apsi_packages/, from the program's .venv of any
    Python version. The result runs on Apsi alone: no venv, no pip. Compiled libraries (C/C++: numpy, pyarrow,
    torch …) cannot be translated: they stay Python imports and the notes name Apsi's own tool for the job.
  • apsi convert keeps Python's semantics byte for byte. Converted files start with use pycompat, which gives
    them Python's object model, scoping and built-ins. Real classes (inheritance from any library class, metaclasses,
    descriptors, __slots__, __init_subclass__, __class_getitem__, name mangling of __private names),
    dataclasses, enums, with, try/except/else/finally, raise … from, generators with send/throw, closures,
    global/nonlocal, chained comparisons, walrus, star targets, del, f-strings with every conversion and nested
    spec — all with Python's behaviour.
  • Every Python 3.13 construct converts: match/case (all pattern kinds and guards), except* and exception
    groups, async def/await/async for/async with and async comprehensions, comprehensions that assign to
    attributes or indexes, type X = … and generic functions/classes (def f[T]), from x import *, a[*idx],
    a[1,], error kinds computed at run time (except KINDS:), names that are Apsi keywords (written in backticks
    and kept as the same name), names read before they are bound (UnboundLocalError, NameError as in Python).
  • Verified on the standard library: 34 pure-Python standard modules (_pydecimal, _pydatetime, argparse,
    tomllib, json, pickle, dataclasses, enum, statistics, ipaddress …, about 38,000 lines) convert and
    print byte-identical output to CPython (tests/test_stdlib_convert.py), alongside 15 feature programs.
  • Faithful conversion is the default (--idiomatic keeps the old mapping onto Apsi built-ins); local modules and
    packages convert with the program; a converted module is reused only when it is newer than its source and was
    written by the same Apsi version; apsi run file.apsi -- args passes sys.argv; SystemExit behaves as in Python.
  • The sandbox stays on: sys, os, pathlib and numpy are permission-checked stand-ins; interpreter internals
    (frames, __globals__, __subclasses__, eval, exec …) need --allow-python=all.

Language

  • New syntax (available everywhere, with Python's meaning): class, with, del, global, nonlocal, match,
    async/await, set literals and comprehensions, generator expressions, is/is not, chained comparisons,
    | & ~ << >> and their assignments, :=, for … else, while … else, bytes literals, \x \u \U escapes,
    positional-only and keyword-only parameters, `name` for identifiers that are keywords.

Real time (use rt)

  • Loops at 1–2 kHz with microsecond lateness (rt.loop, rt.ticks, rt.scheduler for many rates on one loop),
    missed ticks counted and skipped instead of drifting, the garbage collector paused, 1 ns timer slack on Linux and
    1 ms timer resolution on Windows while a loop runs.
  • Video pipelines (rt.video): capture thread, newest-frame-only hand-off, adaptive resolution under sustained
    overload, error isolation, a report with fps, latency, jitter and dropped frames. Any size from 16×16 to
    16384×16384
    with 17 named sizes (vga, hd, hd+, fhd, qhd, 4k …). Measured CPU-only on 2 cores:
    640×480 at 976 fps, 1600×900 at 372 fps, 1920×1080 at 264 fps, 3840×2160 at 65 fps; capped runs hit 240/120/90/60/30
    fps exactly with 1–12 ms latency.
  • Frame helpers on OpenCV, NumPy or the GPU (rt.backend("gpu") through PyTorch or CuPy).

RAM, SSD, CPU and GPU (use memory)

  • memory.array stays in RAM while it fits a budget and moves to a memory-mapped SSD file when it does not;
    memory.chunks and memory.map process data larger than RAM on all cores with read-ahead; memory.store and
    @memory.cached are two-tier caches (RAM, then SSD) with signed entries; memory.fit_batch halves a batch on
    out-of-memory (CPU or CUDA) until it fits; memory.status, memory.devices, memory.best_device.

Hugging Face without Python (use hub)

  • Search, download (cached by commit, resumed, SHA-256 checked, offline mode) and load safetensors / GGUF weights
    from the Hugging Face Hub; access tokens for private and gated models, sent only to the Hub itself.
  • Native tokenizer.json engines: WordPiece (BERT family), byte-level BPE (GPT-2, Llama 3, Qwen),
    SentencePiece-style BPE with byte fallback (Llama 2, Mistral) and Unigram (T5, XLM-R).
  • Graphs that reach far (use layers): graph_transformer (pre-norm residual neighbour attention with
    relations and hop biases), global_attention (one hop to any node of the same graph, landmark keys above
    max_keys), mix_hop (several hop distances in parallel), k_hop shortcut graphs, graph_memory (the k most
    salient node states per graph), graph_transformer_net, hop_distances, eccentricity; graph(…, types=, reverse=true) for relations in both directions. docs/GRAPHS.md.
  • Number formats in native and GPU code. @native and @gpu functions accept f64, f32, f16, bf16, FP8
    (E4M3, E5M2) and int8 tensors. 16- and 8-bit values are read into f32, computed in f32 and rounded once (to
    nearest, ties to even) when stored; the conversions are bit-identical to NumPy and ml_dtypes for every 16-bit
    and 8-bit pattern and work the same on every CPU and GPU. var acc: f32 = 0.0 types a scalar's precision;
    zeros(n, dtype=bf16) and dtype=fp8_e4m3 create real tensors (ml_dtypes). docs/NUMBERS.md.
  • GPU. to_gpu(x) keeps tensors in GPU memory between calls (no copy per call; to_host brings them back). One thread per element for row loops with an independent inner loop (collapse); fewer than 4,096
    iterations run on the CPU and larger sizes are timed once on each and then use the faster; matrix-product loop
    nests run as one cuBLAS GEMM when cuBLAS is installed (Tensor Cores with f32 accumulation for f16/bf16).
  • Exact outer-loop vectorisation. Independent outer iterations with a reduction along the inner loop run four
    at a time with bit-identical results (n-body forces, k-means assignment): n-body 2.8× faster relative to C,
    now ahead of Rust; k-means now faster than C. APSI_JAM=0 turns it off.
  • Platforms. The native compiler works on ARM64 (Linux and Apple silicon); every push is tested on linux-x64,
    linux-arm64, windows-x64, darwin-x64 and darwin-arm64.
  • Faster ordinary code. Struct construction 9× faster (type checkers are built once per annotation, not per
    object); Int/Float argument and return checks, and if/while conditions on plain booleans, skip the checker
    call (recursive calls 2.3× faster); attribute access on structs, lists, dicts and strings takes a short path.
    Measured against 2.0.0 on the same machine: benchmarks/versions (geometric mean 3.83× faster on a
    GitHub runner with the native compiler, 7.31× on a 2-core VM without it).
  • Matrix products written as loops run with BLAS. Three nested loops computing c[i, j] += a[i, k] * b[k, j]
    (any loop order, a hoisted a[i, k], or the dot-product form with an accumulator) are recognised in the compiled
    engine and in @native code and run with the tuned BLAS kernel: 7–10× faster than the same loops in C with -O3,
    about 1,000× faster than before in the compiled engine. The loops remain the fallback (integer arrays, aliasing,
    out-of-range bounds) so behaviour never changes; APSI_IDIOMS=0 turns it off. benchmarks/matmul.
  • Models run natively (hub.model, hub.pipeline): BERT, DistilBERT, RoBERTa, XLM-R and sentence-transformers
    (embeddings, similarity, classification, named entities, fill-mask); GPT-2, Llama, Mistral, Qwen 2/3 and Gemma
    (text generation with a key/value cache, greedy or sampled, streaming).

Fewer AI tokens (use tokens)

  • Exactly reversible transforms for what you send to language models: pack/unpack (lists of records as
    tables: 57–63 % fewer tokens than JSON), dedupe/expand (repeated lines once: 59 % on a log), diff/patch
    (only the changed lines: 99.7 % on an edited document), best, and count (exact with tiktoken).

Data, da...

Read more

Apsi 2.0.0

Choose a tag to compare

@github-actions github-actions released this 21 Nov 18:29

Apsi 2.0 replaces "everything runs on the Python VM" with a native compiler. Python stays where it helps (the
tooling, libraries, the rest of the language), while numeric code runs as machine code built through LLVM.

Python compatibility — write Apsi the way you write Python

  • Exceptions: try / except Kind, Other as e / else / finally and raise (with Python's kind names; errors Apsi
    raises itself — division by zero, missing keys, bad indices, unit mismatches — match them too). Own error types
    are structs with a message field.
  • Generators: yield and yield from produce values lazily; endless generators work.
  • *args / **kwargs in definitions and *xs / **d in calls.
  • Operators on structs: methods add sub mul div eq lt len get contains iter to_str call hash … are used by
    + - * / == < len() [] in for print() … (the conversion of Python's special methods).
  • Pure standard modules without permissions: pyimport("collections"), heapq, itertools, functools, bisect,
    re, string, copy, datetime, fractions, decimal and more need no --allow-python (only public, non-module
    members are exposed, so nothing reaches files, network or processes).
  • New built-ins bor band bxor shl shr bnot (Python's | & ^ << >> ~ for ints, sets, dicts and boolean tensors),
    chain_lists, set, frozenset; json_str(…, indent=, sort_keys=); named tuples print as Point(x=1, y=2).
  • apsi convert keeps behaviour instead of commenting it out: try/except blocks, raise, generators, variadic
    functions, for/while … else, class attributes, special methods, field(default_factory=…), nested
    comprehensions, filter(None, …), sum(xs, start), nonlocal. tests/test_python_compat.py converts real
    Python programs and requires identical output.

Website commands (from any computer)

  • apsi release upload FILE… --version V (platform detected from the file name, resumable 32 MB pieces),
    apsi release list / delete ID / hide / unhide, apsi release bench results.json --version V.
  • apsi yank NAME VERSION [--undo], apsi unpublish NAME VERSION.
  • benchmarks/heavy/run.py: one command, live-updating table in the terminal, --publish V updates
    apsi.dev/benchmarks while it runs, --apsi-root measures another Apsi version.

Native compiler (@native) — new architecture

  • Checked AST → specialised, unit-erased LLVM IR → machine code for the host CPU (-O3, loop and SLP
    vectorisation, AVX2 / AVX-512 / FMA / NEON as available). Each function is specialised on the types, tensor
    dtypes/ranks/layouts and physical units of its arguments; types are inferred by abstract execution.
  • Units at zero cost: dimensional analysis happens at compile time (wrong units are a UnitError), the machine
    code works on SI numbers, and results get their units back. Argument units are still checked on every call.
  • Safe by default: 64-bit integer overflow, index range and integer/float floor-division-by-zero checks with
    source lines; Python semantics for //, %, negative indices, min/max, round. Strict IEEE arithmetic, so
    results are bit-identical to the bytecode engine. Index checks are hoisted out of inner loops by proving
    their range before the loop (the loop is versioned, behaviour unchanged), which lets LLVM vectorise.
  • @native("fast"): checks off, nsw integers, fast-math — for proven code. @native("strict"): an error
    instead of a fallback.
  • Tensors: direct element access on contiguous and strided tensors, row and slice views without copies,
    fused element-wise expressions with NumPy broadcasting (one pass, no temporaries), fused reductions
    (sum mean min max argmin argmax prod dot norm), zeros/ones/full/*_like, zero-copy tensor returns, and
    per-iteration memory arenas so temporaries created in loops are freed every iteration.
  • Never breaks a program: unsupported constructs or argument kinds fall back to the bytecode engine with one
    note naming the construct and line; grad of a native function, evolve calling native right-hand sides, and
    native ↔ bytecode calls all work.
  • Signed machine-code cache (~/.apsi/cache/native, HMAC-SHA256 with the per-user key), keyed by IR, CPU,
    CPU features, LLVM and Apsi versions.
  • Benchmarks (benchmarks/, docs/NATIVE.md): 100–3,500× faster than 1.10;
    @native("fast") fastest or tied single-threaded on Mandelbrot, spectral-norm, sieve and fused-array sums
    against GCC, G++, Clang, Rust and Go, within 6–20 % on n-body and matmul.

Exact semantics

  • Native code now follows the bytecode engine exactly, including where Python numbers and NumPy scalars
    (tensor elements) obey different rules (division by zero, Int overflow, // and % by zero, 0.0 ** -1): every
    value's origin is tracked at compile time, or with a one-bit run-time flag when it depends on the path taken,
    through variables, branches, min/max and function returns. Results come back as the same Python class.
  • tests/test_native_fuzz.py: seeded differential fuzzing of random programs with Python and NumPy arguments —
    values, errors and result classes must all match (0 differences in 14,400 calls). Bugs it found and that are fixed:
    -0.0 // nan (CPython treats NaN as true), floor/ceil return Ints like the bytecode engine, provenance lost
    through abs and Bool → Int.

Parallelism

  • Native thread runtime: dispatcher and workers in machine code, spinning workers with pause, a seqlock-style
    job hand-off, one cache line per shared field and guided chunk claiming — a parallel region starts in ~5 µs
    instead of ~70 µs; parallel speed-ups improved accordingly (spectral-norm 15.9 → 12.6 ms, matmul 20.9 → 15.0 ms
    on 2 cores). Stress-tested with hundreds of thousands of regions, errors mid-region, nesting, thread-count changes
    and concurrent callers, including under gdb; this found and fixed a use-after-return race in the error path and
    an out-of-bounds read.
  • Call latency from bytecode into native code: ~0.9 µs (was ~8 µs) through a signature cache, a direct typed
    entry point and tensors passed as the NumPy object (struct layout measured and verified at start-up); Ints beyond
    64 bits are routed to the bytecode engine instead of being truncated.
  • parallel for (soft keyword): iterations on all CPU cores in native code; deterministic reductions
    (+= -= *=, x = min(x, …), x = max(x, …)) that give bit-identical results for any thread count; one
    thread wake-up per core per loop (chunks are claimed by native workers with an atomic counter). An ordinary for
    on the other engines, so programs stay portable.
  • Automatic multi-core kernels: fused tensor expressions and sum/mean/prod/min/max over 131,072 or more
    elements run on all cores without any code change, with deterministic chunked combination;
    native_threads(n), APSI_THREADS, apsi run --threads N.

GPUs (@gpu)

  • A parallel for becomes a CUDA kernel: Apsi generates PTX for sm_75 (GTX 1660 / RTX 20xx and newer) with
    LLVM's NVPTX backend and loads it through the CUDA driver API (nvcuda.dll / libcuda.so.1) — no CUDA
    toolkit needed. += reductions use GPU atomics.
  • Multi-GPU: iterations are split across all GPUs (APSI_GPUS to choose); each GPU copies back only the rows it
    wrote. Without a GPU, or for kernels outside the GPU rules, the function runs on all CPU cores instead.
  • gpu_info(); use gpu's gpu_info() now works with only the NVIDIA driver (it required CuPy and crashed without it).

Large models

  • apsi plan / use scale: plans training of 1B–2T-parameter models, dense or mixture-of-experts, BF16 or
    FP8, on 13 GPU types: tensor × pipeline × data-parallel layout, recomputation, memory per GPU, GPU count, nodes,
    days, GPU-hours, dataset size and serving memory, with the standard published accounting
    (docs/SCALE.md).
  • @ in native code runs on the multi-threaded BLAS (the same kernels as NumPy), so matrix-heavy native
    functions get BLAS speed plus fused element-wise work around it.

Science, engineering and medicine

  • New standard modules, all verified against reference implementations (tests/test_science.py):
    • use special: gamma, digamma, beta, error functions and inverses, regularized incomplete gamma/beta and
      inverses, Bessel J and I, elliptic K and E, Legendre/associated Legendre, Hermite, Laguerre, Chebyshev,
      spherical harmonics, ζ, E1. Relative errors are 1e-16 to 1e-13 against SciPy.
    • use stats: 11 distributions (pdf/pmf, cdf, sf, ppf, sample), descriptive statistics, t-tests, ANOVA,
      χ², Fisher's exact test, Mann–Whitney, Wilcoxon, Kruskal–Wallis, exact two-sample KS, Jarque–Bera,
      Pearson/Spearman/Kendall, binomial test, Wilson and Clopper–Pearson intervals, OLS and logistic regression,
      power and sample size (exact noncentral t), bootstrap, permutation tests, Bonferroni/Holm/BH/BY. Results agree
      with SciPy and statsmodels to 1e-10 or better.
    • use signal: Butterworth design (second-order sections), zero-phase filtering with SciPy-identical edges,
      notch, windows, convolution, peak detection (height, distance, prominence), Welch, spectrogram,
      Savitzky–Golay, resampling, Hilbert envelope.
    • use mechanics: Verlet, 4th-order Yoshida and RK4 integration; Hamiltonian and Lagrangian dynamics whose
      equations of motion come from automatic differentiation; N-body gravity with energy and momentum
      tracking; Kepler's equation, orbital elements, Hohmann transfers; inertia tensors, quaternions and torqued
      rigid bodies; normal modes; Euler–Bernoulli beams and plane trusses by finite elements.
    • use pde: heat (Crank–Nicolson, ADI), wave, advection, Poisson by fast sine/Fourier transforms, 2-D
      incompressible Navier–Stokes (pseudo-spectral), conjugate gradients, BiCGS...
Read more

Apsi 1.10.0

Choose a tag to compare

@github-actions github-actions released this 21 Nov 18:29

Website and library commands (added to 1.10.0 for apsi.dev)

  • Libraries from the registry: apsi install NAME (also NAME@^1.2, local .apsil files, [dependencies] of
    apsi.toml + apsi-packages.lock), apsi uninstall, apsi pack, apsi publish (resumable upload for large
    libraries), apsi search, apsi lib list | info | login | logout | registry; installed libraries load with use NAME.
  • Removing things again: apsi yank NAME VERSION [--undo], apsi unpublish NAME VERSION.
  • Releases from any computer (administrators): apsi release upload FILE… --version 1.10.0,
    apsi release list, apsi release delete ID, apsi release hide|unhide ID,
    apsi release bench results.json --version 1.10.0. On the server itself: node scripts/admin.js release ….

Development started on Sunday, 8 November 2026 at about 04:00 (Bangkok time, UTC+7); this release was completed the same morning at about 09:00 — about 5 hours of development.

Command line and REPL

  • New REPL: shell words work as in a terminal (cd C:\Users\me\project, cd -, pwd, ls/dir, clear/cls,
    run app.apsi, exit), any operating-system command with !cmd, and 26 :commands (:help, :vars, :type,
    :doc, :time, :bench, :load, :reload, :run, :check, :test, :fmt, :edit, :paste, :save,
    :history, :ast, :tokens, :reset, :perms, :env, …)
    • automatic multi-line input (open brackets, trailing :, \, =, , or an operator), _ holds the last result
    • persistent history, Tab completion of names, fields, :commands and paths (pyreadline3 on Windows),
      colours (respects NO_COLOR), Ctrl+C cancels the current input instead of quitting, helpful hints after errors
  • Shortcuts: apsi alone starts the REPL (or runs a program piped on stdin), apsi FILE.apsi ARGS runs a file,
    apsi -e "CODE" evaluates code, apsi NAME runs an apsi.toml task, mistyped commands get "did you mean"
    • permission flags may come first and never swallow the file: apsi --allow-python app.apsi,
      apsi run --allow-read app.apsi, apsi --allow-python -e "CODE"
  • Projects: apsi new NAME --template app|lib|physics|ai|server, apsi init, apsi.toml
    ([project] main, [run] profile/allow, [tasks]), apsi task [NAME]; apsi run / apsi test without
    arguments use the project
  • New commands: apsi eval, apsi watch (re-run / re-test / re-check on save), apsi bench, apsi doc,
    apsi completion bash|zsh|fish|powershell, apsi help COMMAND
  • apsi run: program arguments (read them with args()), --time, --watch, - for stdin;
    apsi check and apsi fmt accept folders and default to the current folder; coloured error messages
  • New built-ins: args(), input(prompt), exit(code)
  • apsi build app.apsi makes an executable (app.exe on Windows, app on Linux) containing the program, its
    imports, the parsed program, Apsi as bytecode and the permissions chosen at build time; --standalone (Windows)
    adds its own Python runtime with the packages the program uses, --windowed, --add for data
  • compile cache: every program file is parsed once per process (previously the Guard scan, the checker and the run
    each parsed it) and kept in a signed on-disk cache (~/.apsi/cache/compiled); apsi compile PATHS fills it ahead
    of time, --runtime, --stats, --clear, APSI_NO_CACHE=1

Python to Apsi

  • apsi convert translates Python files and whole folders: let/var inference, block-scope hoisting,
    functions, closures, lambdas, classes and dataclasses (with inheritance), generators, comprehensions, f-strings,
    nested and starred unpacking, with open, try/except, comments and docstrings
  • library mapping for numpy, math, random, json, time, os.path, sys, statistics, PyTorch tensors, torch.nn
    layers (→ use layers) and torch.nn.functional; every other library keeps working through pyimport
  • every place that is not an exact translation is reported with its line; the result is checked automatically;
    --run converts, checks and runs; Apsi: Convert Python File to Apsi in VS Code/Cursor

AI

  • use layers: any architecture as a graph of layers — sequential, residual, parallel, dag (any directed
    acyclic graph with several inputs/outputs), dense, convolution, pooling, LSTM/GRU, multi-head and cross attention,
    transformer blocks, normalisation, dropout, 12 activations
  • graph neural networks: graph(), GCN, GraphSAGE, GAT, GIN, graph pooling and differentiable
    segment_sum/mean/max/softmax (gradients verified against finite differences)
  • model zoo: mlp, resnet_mlp, cnn, lstm_classifier, transformer_lm, transformer_classifier, autoencoder,
    vae, gcn_net, gat_net, gin_net, siamese; fit (mini-batches, graph masks, validation), predict, summary
  • PyTorch backend: every use layers model trains and runs on PyTorch with fit(…, backend="torch") — CPU,
    NVIDIA CUDA, AMD ROCm, Apple MPS, Intel XPU; precisions fp64/fp32/tf32 and bf16/fp16 mixed precision (automatic
    loss scaling), torch.compile, fused attention, cuDNN LSTMs; backend="auto", devices(),
    torch_module(model, params) (a real torch.nn.Module) and params_from_torch; check_torch() compares every
    layer's outputs and gradients with NumPy. Needs --allow-python. Guide: docs/BACKENDS.md
  • apsi pkg install torch-auto (also used by the ai group and the setup wizard) picks the PyTorch build from the
    NVIDIA driver version — CUDA 12.8 / 12.6 / 12.4 / 11.8 — ROCm for AMD on Linux, else CPU;
    explicit builds: apsi pkg install torch-cuda (CUDA 12.6) · torch-cu128 (RTX 50) · torch-cu124 · torch-cu118 · torch (CPU):
    PyTorch is taken from PyTorch's own index (a CPU-only wheel from PyPI can no longer slip in), an installed build is
    kept, a different build replaces it
  • fit mini-batches models with several inputs (dag(inputs=[…]), siamese); new mean_pool(axis) layer
  • new complete reference docs/LAYERS.md (every layer with shapes and parameters, recipes, error messages)

Physics, mathematics, data

  • sci.solve_ivp: implicit, L-stable, adaptive TR-BDF2 for stiff systems and adaptive Dormand–Prince RK45, with units
    and dense output (t_eval); reproduces the Robertson kinetics reference values
  • fsum (correctly rounded sums) and Python-compatible built-ins (any, all, divmod, pow, ord, chr, dict,
    set, tuple, iter, next, format, getattr, hasattr, …)
  • data pipelines: from_jsonl, from_parquet, interleave, window (sliding windows), shard
  • new examples 28_any_architecture and 29_stiff_chemistry

Editors

  • one command for every editor: apsi editors [list|install|uninstall], also run by the setup wizard (component
    Code editor support, listed on the System check page) and undone by the uninstaller
  • VS Code family — VS Code, Cursor, Windsurf, VSCodium, VS Code Insiders, Trae — get the same extension (through
    their command-line tool, or unpacked straight into their extension folder when the tool is missing)
  • Sublime Text package: colours, build system with variants and clickable errors, command palette, comments,
    indentation, Goto Symbol, snippets
  • Neovim / Vim package: filetype, colours, indentation, :make, mappings, help file; the language server starts
    automatically in Neovim
  • Notepad++ user-defined language with Apsi colours
  • JetBrains TextMate bundle, Helix, Zed and Emacs (apsi-mode) configurations
  • tools/build_editors.py keeps the Sublime/TextMate grammar and snippets identical to the VS Code ones

VS Code extension

  • richer grammar: definitions vs. calls, built-ins, parameters, types and generic types, units in their own colour,
    as [unit], ±, derivatives, f-string formats, triple-quoted strings, decorators, TODO, Apsi code blocks in Markdown
  • Apsi Dark / Apsi Light themes, snippets, smarter indentation and folding
  • commands: Run, Run with Arguments, Run Project, Test, Test Project, Check, Watch, Benchmark, Audit, REPL,
    Run Selection in REPL (Shift+Enter), Load File into REPL, Run Project Task, New Project, Show Documentation,
    Run Apsi Command, Restart Language Server; ▶ button, status-bar button, right-click menus, key bindings
  • task provider for apsi.toml tasks and an $apsi problem matcher
  • the file icon is now the Apsi logo letters only, on a transparent background

Installers

  • ApsiSetup-1.10.0-linux.sh: one-file installer for every Linux distribution (and WSL), no root, built offline
    by packaging/make_linux_installer.py: commands, .apsi file type and icons, Run with Apsi, templates, editor
    support, uninstall.sh; uses the system Python and installs numpy privately only when it is missing
  • the wheel is built with the standard library only (packaging/make_wheel.py), so release builds work offline;
    the .deb now carries the release manifest (apsi verify passes)
  • Windows: deep recursion works again — the program thread asked for a 256 MiB stack, which Windows refuses (the
    limit is below 256 MiB), so programs fell back to a 1,000-level limit; now 255 MiB with smaller fall-backs
  • Windows 10/11: ApsiSetup-1.10.0-x64.exe contains Python 3.12 and numpy — nothing else to install, no administrator rights
    • adds apsi to PATH, creates Start-menu entries and registers .apsi files
    • .apsi files: logo icon (letters only, transparent, every standard size 16–256), Run with Apsi, Edit
      (VS Code when installed), Test with Apsi, and New → Apsi source file; Explorer refreshes the icon immediately
    • uninstall from Settings → Apps
  • the setup wizard shows its Preparing Apsi Setup… window immediately and keeps it on top of other windows until the
    wizard is ready, then opens the wizard in its place; a balanced 900 × 610 window; software rend...
Read more