Repository navigation
Releases
PLI 5.6.5
PLI 5.6.5: one local server for text, voice, images and video
PLI 5 turns a Mac Studio into a self-contained AI server: Qwen3.8-27B for chat, code and tool calls, plus voice, transcription, images and video with sound, behind one OpenAI- and Anthropic-compatible endpoint, all on the machine.
New in 5.6.4 and 5.6.5
- Video LoRA in memory, several at once: h3 adds the LoRA to each block's weights as it loads them (
tools/h3c-lora.patchfor h3.c), converted once bytools/pli-lora-natif. No fused copy of 33 to 62 GB per LoRA, up to 4 LoRA in one video. Same prompt and seed: the MP4 is byte-for-byte identical to the one rendered from a fused copy, with no measurable time cost. - Room asked of an outside scheduler: a generation child short of memory can ask a scheduler on the same Mac to pause its own jobs (
place_fichier), and waits up to 120 s. /healthshows every model served: per generation family, the model of the last request and the requests in progress.
What's new in PLI 5 (from 5.6.2 and 5.6.3, never published separately)
- One process, one endpoint:
/v1/chat/completions,/v1/messages,/v1/audio/speech,/v1/audio/transcriptions,/v1/images/generations,/v1/videos. Each generation family runs in its own child process, started on demand and unloaded when idle. - The LLM keeps the right of way: images, voices and videos pause between two steps while the LLM writes (97.7 % of its writing speed kept), and the LLM is capped while a child runs.
- Memory-aware: each model declares what it needs, idle families make room, and a request that does not fit gets a clear 503 instead of sending the Mac into swap.
- Fast restart (
--startup-cache on): the 27B starts in 9 s instead of 71 s, same figures, same text. - Cancellation, a prompt guard for images and video, self-maintenance (
PLI_TASKS), exact drafts (PLI_TREE_EXACT=1). - Security review: no secret in the history (gitleaks), static analysis clean (semgrep
p/default).
Measured on an M1 Ultra 64 GB
- Writing: 111 tok/s on a short prompt, 34 tok/s at 96k tokens.
- A 1024² image in 38 to 44 s (Qwen-Image 2.1 + Lightning); a 22-frame video with sound in 2 to 2.5 minutes.
Before you upgrade
- Pre-release, validated on one machine (Mac Studio M1 Ultra, 64 GB) and one model family (Qwen3.8-27B).
- The package is
pli; saved conversations move to their new key withpython -m pli.additions.rekey. - Generation is off by default (
PLI_GEN=1andPLI_GEN_CONFIG; examples indocs/gen/).
Everything else: CHANGELOG.md.
PLI 4.2.1
- Copy log for head rounds too (
PLI_COPY_LOG=<file>, off by default): a round where a copy was offered but the
learned choice (PLI_HEAD_BEFORE_COPY=3) took the head's tree now writes the copy, the whole tree (tokens, parents)
and the choice's moving average, so the choice can be checked offline against the text written. The choice itself
does not change.PLI_COPY_LOGstays out of the checkpoint key's settings fingerprint. python -m tensorfold.pli.rekey plan-code: the newest checkpoint key with the digest of this tree's code, for a
code change that does not change a checkpoint's bits. First release whose number changes without touching the
checkpoint key (4.2.0 took the release number out of it).
PLI 4.2.0
- Checkpoints written in the background (
PLI_SPILL_BACKGROUND=1): a conversation's checkpoint goes to disk from
its own thread instead of the engine's. Over 24 hours of service on an M1 Ultra the engine waited 143 s for the disk
instead of 424 s the day before (spills 328 -> 88 s). Same text. - Tool calls follow their schema (
PLI_TOOL_GRAMMAR=1, needsxgrammar): when a request offers tools, the reply's
calls are constrained by a grammar built from the tools' JSON schemas (name, required keys, valid values). On 24 agent
turns of a real session, 24 valid calls against 20 without the grammar; same text elsewhere. - End of a reply read back from disk (
PLI_DISK_REPLY_END=1): when the checkpoint of a finished reply was evicted
from memory but is on disk, the next turn resumes from it instead of reading the conversation again (first word of a
resumed turn 1 546 s -> 6.9 s on the bench, 9 texts of 9 identical). - The code enters the checkpoint key (
PLI_CODE_KEY=1): the sources of PLI and of the prompt path are hashed into
the key, so a checkpoint computed by other code is never resumed. With it on, the release number leaves the key: a
new release that does not change the computation keeps its checkpoints. python -m tensorfold.pli.rekeyre-keys checkpoints on disk without recomputing them (header and file name
rewritten, tensors copied, modification time kept), for a key change that does not change their bits. Used to move
the checkpoints of 4.1.2 to the key of 4.2.0.- Request tokens kept for training (
PLI_FALCON_TOKENS=<dir>, off by default): the prompt and reply tokens of each
request, one file per request (directory mode 700), deleted afterPLI_FALCON_TOKENS_DAYSdays (7), for a night job
that recomputes the target's states and adjusts the draft on real traffic. Nothing changes in the computation. - The tool-grammar tests read the tokenizer from
PLI_TEST_QWEN38_MODEL(skipped when absent).
PLI 4.1.2
pli setupturns A1 on. With them1-ultraprofile, the list shown before downloading has one more item: A1's
map (Latent Sparse Attention, released-lsa-weights-v1, 6 MB, Apache-2.0). On yes,pli setupdownloads it from
the GitHub release, checks its SHA-256 (nothing is kept if it differs), converts it to NumPy in<prefix>/models,
and writesPLI_A1_LSA_MAP,PLI_S3B_THRESHOLD=16384andPLI_S3B_ROWS=4096in the service. It is left out with
--no-a1, on other Macs, and when the model'sconfig.jsonis not Qwen3.8-27B's; a failed download leaves PLI
without A1 and the install goes on. Before, A1 had to be set by hand.- The map is converted without torch (
src/tensorfold/install/lsa_map.py, used bytools/convert_lsa_map.py
too): the PyTorch file is read as the zip of arrays it is, with a pickle reader that accepts only the names of a
tensor file and refuses any other, so no code from the file runs. On the release's file it writes the same .npz,
byte for byte, as the torch path of 4.1.1 and the map served in production.
PLI 4.1.1
- Name of A1's map. 4.1.0 named the file to download
qwen38_27b.pt; the file of LSA's release
d-lsa-weights-v1isqwen38_27b__long-16k-1M-heavy-key-and-query-d2048.pt(SHA-25624cee934f969756db228a8df7f2bc7613979b63ac3b1d5ac3d9aa42e4339bb9b). README,
docs/switches.mdandtools/convert_lsa_map.pygive that name and the download command. tools/convert_lsa_map.pychecked: run on that file (torch 2.13.0, NumPy 2.4.6), it writes the 32 arrays (16
layers,W_Q_idxandW_K_idx, [768, 64] float32) equal to the map served in production since 4.1.0's A1.
No code change.
PLI 4.1.0
checkpointfield of/healthand/v1/models. The served weights (folder, or Hugging Face repo and revision),
their model type and quantization, the draft, the MTP head and the PLI version, so a client sees which model answers
without reading the server's settings. Same text.- Checkpoints kept in 4 bits (
PLI_CHECKPOINT_BITS=4in them1-ultraprofile, off in the code; in production
since 4 Nov). A checkpoint evicted from memory or written to disk is
rotated (Hadamard transform, as in QuaRot) and quantized in groups of 64 (PLI_CHECKPOINT_GROUP); on resume it is restored to 16
bits. The checkpoint key carries the format, so a file written in 16 bits is not read as 4 bits, and back. Under
memory pressure (agent sessions that evict their checkpoints): disk of the checkpoints divided by 4, time to the
first token of resumed turns -54%; on other resumes -6 to -13%. Text can change on resumed turns: on conversations
judged turn by turn (code, admin, maths), no right answer became wrong; one maths answer wrong at turn 2 was corrected
at turn 3.8(8 bits, no rotation) is accepted; it was measured and not kept. The production name
PLI_POINTS_BITSis still accepted. - A1 in the prefill (
PLI_A1_LSA_MAP,PLI_A1_PREFILL_ONLY=1; in production since 5 Nov). Past
PLI_S3B_THRESHOLDpositions, each row of a prompt chunk reads the 10% of the tiles of 32 keys that the index of
Latent Sparse Attention scores best, plus the last 1,024
positions; decode and trees keep the attention of 4.0.1. The distilled map is not shipped: downloadqwen38_27b.pt
from LSA's released-lsa-weights-v1(Apache-2.0) and convert it withtools/convert_lsa_map.py(needs torch).
It goes withPLI_S3B_THRESHOLD=16384andPLI_S3B_ROWS=4096in the environment; without the map, the profile's
65,536 stays and nothing changes. On Qwen3.8-27B (Sabi-Noema 4-bit), M1 Ultra: time to the first token at 146k
tokens 712.5 s instead of 844.6 s (-15.6%), -6% at 61k; offline, the NLL of the reduced prefill equals the full
one (0.72229 against 0.72127 at 48k, 1.01024 against 1.0106 at 97k). Text can change: on conversations judged turn
by turn, two answers changed from right to wrong and three got better. Experimental, not in production:
PLI_A1_MAX_TILES(a cap on the tiles per row: -22 to -27% at 146k but the NLL rises, +0.0046 at 160 tiles) and
PLI_A1_PREFILL_ONLY=0(A1 for decode too: decode 5 to 21% slower on long conversations). The production names
PLI_ANCRE1_LSA,PLI_ANCRE1_LECTURE_SEULE,PLI_A1_TRANCHE,PLI_A1_PETITEandPLI_A1_GARDES_MAXare still
accepted.
PLI 4.0.1
pli serveprints the PLI mark (folded paper) with the version when it starts. The README gets the PLI logo,
a speed summary, a Hugging Face link for each model family, and says that PLI is tuned for the M1 Ultra.pli setupinstalls PLI;install.shonly makes the virtual environment../install.sh installfinds Python
3.11 to 3.14, creates~/pli/venv(or--prefix), installs PLI in it and runspli setupwith the same options.
pli setupdoes what the rest ofinstall.shdid, with the same checks, messages, defaults and service settings
(profilem1-ultraon an Ultra chip with 64 GB,PLI_ANE_FFN,MLX_MAX_MB_PER_BUFFER=500,
MLX_MAX_OPS_PER_BUFFER=100,--context 229376 --parallel 4,--spill-gib 40with 70 GB free, the hybrid draft),
throughpli pullandpli service.pli status,start,stop,restart,logsanduninstallreplace the
commands ofinstall.sh, which still run them (./install.sh statusispli status).- Nothing is downloaded without consent. Before downloading,
pli setuplists each item: Hugging Face repo, page
and revision (main, not pinned, with its commit), license from the model card, size read from the Hub's API
("about" when the Hub does not answer), where it goes, the free disk, and what it brings. Files already in the
Hugging Face cache are marked "already downloaded" and not counted. It then asks[y/N];--yesanswers yes, and
without a terminal and without--yesit stops and says what to add. It stops too when the disk is too small. - Each item is optional.
--no-hybrid(no MTP head),--no-drafter(new: no draft at all,--drafter none --no-drafts),--no-models(nothing downloaded; the service now starts when the model and the draft are already on
the Mac),--model DIR,--drafter REPO|DIR(new) and--mtp-head FILE(new: a head already written). pli benchruns the public suite pli-bench-v1 against the local server (the port ofpli setup), thinking off.
The bench and its texts (811 KB) are now part of the package (src/tensorfold/bench/);tools/pli_bench.pyruns the
same code from a clone, with standard Python only.tools/quantize_mtp_head.pylikewise runs
tensorfold.install.mtp_head, whichpli setupuses.plialone prints a short help with examples instead of an argument error.- Checkpoint at the start of the last message (
PLI_LAST_MESSAGE_CHECKPOINT, 1,024 in them1-ultraprofile,
off in the code; in production since 3 Nov). Besides the history checkpoint, PLI keeps the state at the start of
the prompt's last message (the turn marker that opens the reply, looked for backwards in the history) when at least
N tokens are read there since the resume. The next request that keeps the history but not that message (a question
replaced, a reply thrown away, a sub-agent on the same history) starts from there instead of from the previous
request's checkpoint. Same text. On the agent session of the public suite pli-bench-v1: the same 8 replies, prompt
served from the cache 73.5% instead of 58.9%, sum of the times to the first token 238.0 s instead of 357.3 s
(-33%), writing speed unchanged. The production namePLI_DEBUT_DERNIERis still accepted.
PLI 4.0.0
PLI 4.0.0 is a pre-release. It is validated on one machine (Mac Studio M1 Ultra, 64 GB, macOS 27) and one model
family (Qwen3.8-27B dense, MLX 4-bit). Other Macs and models run, but are not measured.
PLI is an LLM inference server for Apple Silicon, forked from TensorFold
0.6.6 (Apache-2.0). It keeps TensorFold's OpenAI- and Anthropic-compatible API and its exact speculative decoding, and
adds work that spreads Qwen3.8-27B across the GPU, the CPU and the Neural Engine, its own prompt and decode kernels,
and a draft that picks, per conversation, between DFlash2 and the model's own MTP head.
Since 3.6.0, the last public version (details in the CHANGELOG, sections 3.7.0 and 4.0.0):
- Drafts: hybrid DFlash2 + MTP head chosen per conversation, DFlash2 alone for one-turn questions, draft trees for
one-turn questions, copy of the context or DFlash2 chain chosen per stream, extended draft lists, faster draft
kernels. - Verify and decode: Kapitsa's 8-row verify GEMM,
attmon short rows, reduced attention anchored to positions, and
for small prompt chunks beyond 65,536 tokens of context. - Neural Engine: weights requantized on the CPU, kept awake between layers, units chosen by chunk size (private Apple
APIs: a macOS update can break it; PLI then computes on the GPU). - Agent turns: the reply kept for the next turn, checkpoints without a copy of the KV cache, checkpoints at fork
points, evicted checkpoints written to disk, control tokens in a request kept as text. --profile m1-ultrasets every production setting;install.shinstalls PLI as a per-user service with its models.- A public bench with fixed prompts (
tools/pli_bench.py): run it on your Mac and compare. - Settings renamed in English; the older names are still accepted.
Measured with the public suite on the validated machine (prompt reading / writing, tokens per second):
| Prompt | PLI 4.0.0 | Kapitsa 0.2.24 | llama.cpp b11926 | mlx-lm 0.32.2 |
|---|---|---|---|---|
| 1k | 294 / 53.2 | 210 / 65.3 | 193 / 25.1 | 159 / 24.0 |
| 32k | 285 / 49.2 | 189 / 43.1 | 168 / 19.9 | 153 / 18.8 |
| 64k | 245 / 40.8 | 167 / 32.9 | 144 / 17.1 | 136 / 17.2 |
| 96k | 213 / 34.1 | 151 / 33.6 | 126 / 15.5 | not measured |
| agent session, writing (first 6 answers) | 47.0 | 46.7 | 24.3 | 20.9 |
Setup, every size and the agent session: README, section Measurements. Every change and its measurement:
CHANGELOG.md. Install: git clone https://github.com/nicopingu/pli.git && cd pli && ./install.sh install.