You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
2.4.4 — Python, exactly; real time; data, databases and the web
AI training on your own GPU — faster than PyTorch, nothing else to install
fit(…, backend="gpu") trains with kernels Apsi generates itself from the NVIDIA display driver alone: tiled
tensor-core matrix products with the bias, activation and residual fused in, fused attention, one-kernel AdamW, the
data set kept on the card. Measured on an RTX 3060, same model and same data: 151,744 tokens/s against PyTorch
2.14 bfloat16's 126,993 (1.19×), every epoch at the same speed (153,552 · 154,009 · 151,667), the card held near
77 °C. fp8 144,176, fp4 142,284 tokens/s. A 127 M-parameter GPT: 18,074 tokens/s held at 77 °C.
Four builds of every matrix product, timed by the card on the real shapes the first time; the fastest is kept.
Several cards:device="cuda:all" — reduce-scatter, sharded optimizer state, all-gather; identical results
to one card with the whole batch.
Steady temperature without pauses: the card's power limit is moved to hold 77 °C. Set it up once with the Apsi GPU Tuner (download page: Apply / Restore, Windows and Linux) or apsi gpu setup; apsi gpu hold 77
holds the temperature for any program.
ema= and ewc= for smoother weights and continual learning; low-bit training in fp8, fp6, fp4, int8, int4.
apsi serve-model: the OpenAI API with streaming, a fair queue and metrics; 0.5B, 3B and 14B models fully on
a 12 GB card.
Python conversion: exact, complete
Libraries come along: the installed libraries a program imports that are written in Python (requests, urllib3, idna, certifi …) are converted with it into apsi_packages/, from the program's .venv of any
Python version. The result runs on Apsi alone: no venv, no pip. Compiled libraries (C/C++: numpy, pyarrow, torch …) cannot be translated: they stay Python imports and the notes name Apsi's own tool for the job.
apsi convert keeps Python's semantics byte for byte. Converted files start with use pycompat, which gives
them Python's object model, scoping and built-ins. Real classes (inheritance from any library class, metaclasses,
descriptors, __slots__, __init_subclass__, __class_getitem__, name mangling of __private names),
dataclasses, enums, with, try/except/else/finally, raise … from, generators with send/throw, closures, global/nonlocal, chained comparisons, walrus, star targets, del, f-strings with every conversion and nested
spec — all with Python's behaviour.
Every Python 3.13 construct converts:match/case (all pattern kinds and guards), except* and exception
groups, async def/await/async for/async with and async comprehensions, comprehensions that assign to
attributes or indexes, type X = … and generic functions/classes (def f[T]), from x import *, a[*idx], a[1,], error kinds computed at run time (except KINDS:), names that are Apsi keywords (written in backticks
and kept as the same name), names read before they are bound (UnboundLocalError, NameError as in Python).
Verified on the standard library: 34 pure-Python standard modules (_pydecimal, _pydatetime, argparse, tomllib, json, pickle, dataclasses, enum, statistics, ipaddress …, about 38,000 lines) convert and
print byte-identical output to CPython (tests/test_stdlib_convert.py), alongside 15 feature programs.
Faithful conversion is the default (--idiomatic keeps the old mapping onto Apsi built-ins); local modules and
packages convert with the program; a converted module is reused only when it is newer than its source and was
written by the same Apsi version; apsi run file.apsi -- args passes sys.argv; SystemExit behaves as in Python.
The sandbox stays on: sys, os, pathlib and numpy are permission-checked stand-ins; interpreter internals
(frames, __globals__, __subclasses__, eval, exec …) need --allow-python=all.
Language
New syntax (available everywhere, with Python's meaning): class, with, del, global, nonlocal, match, async/await, set literals and comprehensions, generator expressions, is/is not, chained comparisons, | & ~ << >> and their assignments, :=, for … else, while … else, bytes literals, \x \u \U escapes,
positional-only and keyword-only parameters, `name` for identifiers that are keywords.
Real time (use rt)
Loops at 1–2 kHz with microsecond lateness (rt.loop, rt.ticks, rt.scheduler for many rates on one loop),
missed ticks counted and skipped instead of drifting, the garbage collector paused, 1 ns timer slack on Linux and
1 ms timer resolution on Windows while a loop runs.
Video pipelines (rt.video): capture thread, newest-frame-only hand-off, adaptive resolution under sustained
overload, error isolation, a report with fps, latency, jitter and dropped frames. Any size from 16×16 to
16384×16384 with 17 named sizes (vga, hd, hd+, fhd, qhd, 4k …). Measured CPU-only on 2 cores:
640×480 at 976 fps, 1600×900 at 372 fps, 1920×1080 at 264 fps, 3840×2160 at 65 fps; capped runs hit 240/120/90/60/30
fps exactly with 1–12 ms latency.
Frame helpers on OpenCV, NumPy or the GPU (rt.backend("gpu") through PyTorch or CuPy).
RAM, SSD, CPU and GPU (use memory)
memory.array stays in RAM while it fits a budget and moves to a memory-mapped SSD file when it does not; memory.chunks and memory.map process data larger than RAM on all cores with read-ahead; memory.store and @memory.cached are two-tier caches (RAM, then SSD) with signed entries; memory.fit_batch halves a batch on
out-of-memory (CPU or CUDA) until it fits; memory.status, memory.devices, memory.best_device.
Hugging Face without Python (use hub)
Search, download (cached by commit, resumed, SHA-256 checked, offline mode) and load safetensors / GGUF weights
from the Hugging Face Hub; access tokens for private and gated models, sent only to the Hub itself.
Graphs that reach far (use layers): graph_transformer (pre-norm residual neighbour attention with
relations and hop biases), global_attention (one hop to any node of the same graph, landmark keys above max_keys), mix_hop (several hop distances in parallel), k_hop shortcut graphs, graph_memory (the k most
salient node states per graph), graph_transformer_net, hop_distances, eccentricity; graph(…, types=, reverse=true) for relations in both directions. docs/GRAPHS.md.
Number formats in native and GPU code.@native and @gpu functions accept f64, f32, f16, bf16, FP8
(E4M3, E5M2) and int8 tensors. 16- and 8-bit values are read into f32, computed in f32 and rounded once (to
nearest, ties to even) when stored; the conversions are bit-identical to NumPy and ml_dtypes for every 16-bit
and 8-bit pattern and work the same on every CPU and GPU. var acc: f32 = 0.0 types a scalar's precision; zeros(n, dtype=bf16) and dtype=fp8_e4m3 create real tensors (ml_dtypes). docs/NUMBERS.md.
GPU.to_gpu(x) keeps tensors in GPU memory between calls (no copy per call; to_host brings them back). One thread per element for row loops with an independent inner loop (collapse); fewer than 4,096
iterations run on the CPU and larger sizes are timed once on each and then use the faster; matrix-product loop
nests run as one cuBLAS GEMM when cuBLAS is installed (Tensor Cores with f32 accumulation for f16/bf16).
Exact outer-loop vectorisation. Independent outer iterations with a reduction along the inner loop run four
at a time with bit-identical results (n-body forces, k-means assignment): n-body 2.8× faster relative to C,
now ahead of Rust; k-means now faster than C. APSI_JAM=0 turns it off.
Platforms. The native compiler works on ARM64 (Linux and Apple silicon); every push is tested on linux-x64,
linux-arm64, windows-x64, darwin-x64 and darwin-arm64.
Faster ordinary code. Struct construction 9× faster (type checkers are built once per annotation, not per
object); Int/Float argument and return checks, and if/while conditions on plain booleans, skip the checker
call (recursive calls 2.3× faster); attribute access on structs, lists, dicts and strings takes a short path.
Measured against 2.0.0 on the same machine: benchmarks/versions (geometric mean 3.83× faster on a
GitHub runner with the native compiler, 7.31× on a 2-core VM without it).
Matrix products written as loops run with BLAS. Three nested loops computing c[i, j] += a[i, k] * b[k, j]
(any loop order, a hoisted a[i, k], or the dot-product form with an accumulator) are recognised in the compiled
engine and in @native code and run with the tuned BLAS kernel: 7–10× faster than the same loops in C with -O3,
about 1,000× faster than before in the compiled engine. The loops remain the fallback (integer arrays, aliasing,
out-of-range bounds) so behaviour never changes; APSI_IDIOMS=0 turns it off. benchmarks/matmul.
Models run natively (hub.model, hub.pipeline): BERT, DistilBERT, RoBERTa, XLM-R and sentence-transformers
(embeddings, similarity, classification, named entities, fill-mask); GPT-2, Llama, Mistral, Qwen 2/3 and Gemma
(text generation with a key/value cache, greedy or sampled, streaming).
Fewer AI tokens (use tokens)
Exactly reversible transforms for what you send to language models: pack/unpack (lists of records as
tables: 57–63 % fewer tokens than JSON), dedupe/expand (repeated lines once: 59 % on a log), diff/patch
(only the changed lines: 99.7 % on an edited document), best, and count (exact with tiktoken).
Data, databases, web
use dataframe: a columnar table engine (filter, select, group by, join, sort, windows, CSV/JSON/JSONL, a C
CSV parser). Against pandas on 2M rows: filters 24×, group-by 4.2–4.3×, joins 3.1×, sorting 1.8× faster, 4× less memory.
use db: PostgreSQL (v3 protocol, SCRAM, TLS, parameters, prepared statements, LISTEN/NOTIFY, pools),
MySQL / MariaDB (caching_sha2 and native authentication, TLS), MongoDB (OP_MSG, BSON with decimal128,
SCRAM-SHA-256/1, cursors), Redis (pipelines, pub/sub) and SQLite — no driver to install.
Data-parallel training:fit(…, workers=N) splits every batch across N processes and averages the gradients
(the same step as one process).
WebSocket server routes, rooms and a client (app.websocket, web.websocket_connect), RFC 6455 complete.
The HTTP server is about 3× faster (an asyncio protocol; routes that prove fast run on the event loop, measured
by CPU time so a busy server never misleads it; slow or blocking routes stay on the workers): 20,300–20,700
requests per second from 50 to 1,000 concurrent connections on 2 cores, up from 6,700–7,700.
use env: .env files, typed settings and secrets that print as ******; apsi run --env-file, --venv
(or a .venv next to the program) for pyimport.
Tools
Apsi License 2.0 replaces the Non-Commercial License 1.0: any use is allowed, including commercial work;
only selling Apsi itself needs a licence. docs/LICENSING.md explains it with examples.
The formatter keeps hand-aligned continuation lines and comment columns; @native and @gpu functions run as
ordinary functions in the interpreter engine instead of stopping it.
Debugger: apsi debug in the terminal and F5 in VS Code-family editors (apsi dap): breakpoints with
conditions, hit counts and log points, stepping, the call stack, variables, tensor summaries, stop on errors.
VS Code extension: finds every installed Apsi version (the one chosen with apsi use first) and runs its
Python directly, so the language server and Convert Python File to Apsi work with current VS Code on Windows;
a failed conversion says why.
Faster programs: the main program starts with the loaded runtime frozen out of the cycle collector and a
50,000-allocation first generation: object-heavy code runs about 8 % faster than on CPython (APSI_GC=default
turns it off). Module attribute access (math.sqrt) is a plain attribute lookup.
Faster start and compile: start-up about 30 % faster (heavy modules load on first use); a cold compile of a 20,000-line program 3.7 → 2.5 s (faster
lexer and parser, the Guard scan 3× faster, a signed "checked clean" marker so unchanged programs are not checked
again).
Faster web server
Less work per request (request ids without a system call, merged security headers, literal routes without a
regular expression, one timer per connection, one-regex header validation, fast routes timed by the wall clock
only): 19,200 → 22,400 requests/s per process on a 2-vCPU VM.
uvloop when installed (pip install "apsi[web]"); app.run(processes="auto"); stop() also ends uvloop's serve_forever. Synthetic test on a 4-vCPU EPYC 7763: 1 process 14,600–15,700 req/s, processes="auto"
38,800–41,500, Node.js 22 (bare http, one process) 23,400–24,100.
Syntax and library
a ?? b, a?.b, a?.f(x), a?[i] in both engines, the checker, the formatter and the editor grammars.
Container-aware: the memory cgroup's limit (v1 and v2) is used when it is smaller than the machine's, for the
budget, array(), status() and the new limits().
pressure() (ok / moderate / high / critical, from free memory and Linux PSI); under high pressure big arrays go
to the SSD and stores and pools give RAM back by themselves; relieve(); on_pressure(fn) watcher.
pool(): reusable buffers for big temporaries (take / give), 1.36× faster for 256 MB temporaries.
chunks() and map() stream memory-mapped files with flat memory (3 GB file: peak RSS 3,103 → 97 MB, same speed),
with kernel read-ahead; chunks() of arrays in RAM returns views.
Stores keep a running SSD total: set no longer lists the folder (8.6 → 0.07 ms per set at 8,000 entries).
report(): every array from array(), its size and whether it is in RAM or on the SSD.
Speed on every processor
The jammed lanes of an outer loop (n-body forces, matrix-vector products, distances) are emitted as SIMD vector
instructions by Apsi itself instead of relying on LLVM's SLP vectoriser, which with LLVM 22 failed on AVX2
processors: n-body 127.5 → 46.0 ms on an EPYC 7763 (gcc -O3: 44.2 ms); results bit-for-bit unchanged (tests/test_jam.py).
@native("fma"): checks on, a * b + c may fuse into one instruction (gcc's default for C); ODE benchmark
440 → 288 ms (gcc -O3: 387 ms).
@gpu CPU/GPU choice: the first GPU launch (kernel loading) is no longer counted, and the losing side is timed
again every 64 calls (tests/test_gpu_choice.py).
benchmarks/commits/measure.py and the commits workflow measure several commits on one machine, in rotating
rounds; the native cache now also depends on APSI_SLP. From the start of this work to the release, on one
EPYC 7763: n-body 4.0×, ODE 1.5×, k-means 1.27×, compiling 1.21× faster, nothing slower
(benchmarks/commits/results-2.4.4.md).