Skip to content

PLI 5.6.5

Pre-release
Pre-release

Choose a tag to compare

@nicopingu nicopingu released this 17 Nov 21:04

PLI 5.6.5: one local server for text, voice, images and video

PLI 5 turns a Mac Studio into a self-contained AI server: Qwen3.8-27B for chat, code and tool calls, plus voice, transcription, images and video with sound, behind one OpenAI- and Anthropic-compatible endpoint, all on the machine.

New in 5.6.4 and 5.6.5

  • Video LoRA in memory, several at once: h3 adds the LoRA to each block's weights as it loads them (tools/h3c-lora.patch for h3.c), converted once by tools/pli-lora-natif. No fused copy of 33 to 62 GB per LoRA, up to 4 LoRA in one video. Same prompt and seed: the MP4 is byte-for-byte identical to the one rendered from a fused copy, with no measurable time cost.
  • Room asked of an outside scheduler: a generation child short of memory can ask a scheduler on the same Mac to pause its own jobs (place_fichier), and waits up to 120 s.
  • /health shows every model served: per generation family, the model of the last request and the requests in progress.

What's new in PLI 5 (from 5.6.2 and 5.6.3, never published separately)

  • One process, one endpoint: /v1/chat/completions, /v1/messages, /v1/audio/speech, /v1/audio/transcriptions, /v1/images/generations, /v1/videos. Each generation family runs in its own child process, started on demand and unloaded when idle.
  • The LLM keeps the right of way: images, voices and videos pause between two steps while the LLM writes (97.7 % of its writing speed kept), and the LLM is capped while a child runs.
  • Memory-aware: each model declares what it needs, idle families make room, and a request that does not fit gets a clear 503 instead of sending the Mac into swap.
  • Fast restart (--startup-cache on): the 27B starts in 9 s instead of 71 s, same figures, same text.
  • Cancellation, a prompt guard for images and video, self-maintenance (PLI_TASKS), exact drafts (PLI_TREE_EXACT=1).
  • Security review: no secret in the history (gitleaks), static analysis clean (semgrep p/default).

Measured on an M1 Ultra 64 GB

  • Writing: 111 tok/s on a short prompt, 34 tok/s at 96k tokens.
  • A 1024² image in 38 to 44 s (Qwen-Image 2.1 + Lightning); a 22-frame video with sound in 2 to 2.5 minutes.

Before you upgrade

  • Pre-release, validated on one machine (Mac Studio M1 Ultra, 64 GB) and one model family (Qwen3.8-27B).
  • The package is pli; saved conversations move to their new key with python -m pli.additions.rekey.
  • Generation is off by default (PLI_GEN=1 and PLI_GEN_CONFIG; examples in docs/gen/).

Everything else: CHANGELOG.md.