Skip to content

mlx-agent v0.6.0

Choose a tag to compare

@github-actions github-actions released this 08 Oct 14:10
· 368 commits to main since this release
v0.6.0
b95e972
  • Add serve start --jit: the loopback endpoint stays reachable while its local model is unloaded; chat and completion requests load one owned worker from the confirmed local files, concurrent requests share the load, and active streams block unload.

  • Add serve unload (receipt-fenced, --expected-pid) to release JIT weights while the endpoint stays reachable, and separate residency in serve status.

  • Add JIT memory policy: --idle-timeout, --keep-loaded, and --min-headroom-gb on serve start and serve policy; serve policy updates a running endpoint without a restart; cold loads check fresh OS headroom against estimated weights, runtime allowance, and reserve, and unknown measurements block loading.

  • Fix: convert describe decodes the answer with the tokenizer's special tokens skipped; moondream3 answers no longer start with <|md_reserved_4|>.

  • Fix: convert transcribe with Whisper vocabularies before large-v3 ends at the end of speech. mlx-audio resolved their missing <|nospeech|> to <|endoftext|> and suppressed it, so every transcript ran to the token limit with invented text; the runner uses <|nocaptions|> instead.

  • Add convert speak: synthesize one text with a converted text-to-speech model to a new WAV (voice, speed, language), returning audio length, synthesis time, real-time factor, and peak memory (read-only).

  • Add convert describe: answer one question about an image or a video with a converted vision-language model through its backend, returning token counts, throughput, and timing (read-only).

  • Add convert video: render one prompt with a converted text-to-video model to a new MP4 (size, frames, fps, steps, seed), returning frame count, duration, generation and load time, seconds per frame, peak memory, and pixel spread (read-only).

  • Add the mlx-video backend (hash-locked mlx-video-with-audio 0.1.37) with a t2v port: intake resolve and convert start take the original Wan2.1 T2V checkpoint (Wan-AI/Wan2.1-T2V-1.3B) to an MLX directory at 4 or 8 bits; new task type and backend category video_generation (pipeline tag text-to-video).

  • Add a DeepSeek-V4 DSpark drafter port (deepseek_v4_dspark): convert start --gguf on llama.cpp's dflash drafter GGUF writes an MLX sidecar, tensors named as DeepSeek's checkpoint (mtp.N as stages.N), FP8-sourced projections affine-quantized, MXFP4 routed experts repacked to MLX mxfp4 exactly, config.json with the DSpark fields and text_config. dspark-DeepSeek-V4-Flash-0731-Q8_0 at 4 bits: 4,730 tensors, 10.67 GB; F32, BF16, and FP4 tensors equal DeepSeek-V4-Flash-0731's own safetensors.

  • convert scan labels speculative-decoding drafters (dflash, eagle3 GGUFs; configs with dspark_target_layer_ids) speculative_draft (use case speculative_decoding), reports draft (port, target, block_size), and keys a drafter apart from its target so it is never grouped as the target's duplicate. Drafters no port converts are refused at plan time (unsupported_draft).

  • Committed provider packages hold one runtime per installed package instead of one per skill: tracked files under providers/ drop from 1871 to about 200. providers/claude/ is removed (Claude Code and the installer's claude provider use the repository-root plugin files); providers/codex and providers/agy carry one runtime at the package root, which their skills resolve as <plugin-root>/scripts/mlx-agent; providers/opencode and providers/agentskills carry none.

  • install agentskills copies the root launcher and runtime into each installed skill, and install opencode copies root src/mlx_agent; the cp -R providers/agentskills/... route is removed from the docs.

  • update and uninstall remove receipt-owned files that the current manifest no longer declares (the per-skill runtime copies of earlier releases), refusing any whose content changed. Transaction changes accept remove, receipted with a backup and restored by rollback.

  • Scout command and skill text point at the reference packs under the package runtime root (${CLAUDE_PLUGIN_ROOT}/src/..., <plugin-root>/src/..., <skill-dir>/src/..., <config-dir>/src/...) instead of a bare src/....

  • Fix: the OpenCode mlx_agent_command tool accepts all seven capabilities (it accepted only scout, adopt, and wire while bench, doctor, watch, and fleet commands called it).

  • Fix: the Codex plugin author is generated from the manifest publisher (organization_url) instead of a hardcoded GitHub profile.

  • Add the mflux backend (0.20.0, hash-locked) with an image_generation category and a Qwen-Image 2.1 port (qwen_image_21): its converter merges an optional PEFT LoRA into the transformer (W + scale·B@A, rounded to the weight dtype), quantizes with mflux, saves mflux's component layout, and writes a root config.json (components, quantization). Diffusers repos resolve by model_index.json class (port_pipelines); curated port_recipes name a repository's pinned base plus its LoRA (abenzerps/Qwen-Image-2.1-Uncensored-GGUF: Qwen/Qwen-Image-2.1@d26bb61 + its LoRA at 1.0, equal bit for bit to its BF16 GGUF on the checked tensors). intake fetch downloads a recipe's base snapshot and LoRA; convert start reads both from the cache. intake resolve reports recipe.

  • Add convert generate: render one prompt with a converted image-generation model to a new PNG (size, steps, seed).

  • convert scan lists outputs saved as per-component folders when config.json names the components.

  • Add the mlx-embeddings backend (0.1.0, hash-locked) with a classification category, and a Laya port (laya, convaiinnovations/laya root checkpoint): ModernBERT encoder plus the option-marker decision head, with the shipped temperatures. Repos without a root config.json resolve through a port's port_signatures; convert start --model-type runs a port's own converter (port_convert); port_quantize takes a group_size (Laya: encoder and head linears at group 32).

  • Subfolder checkpoints: org/name/folder and tree/<rev>/<folder> links resolve against that folder's files (config, weights headers, port signature, download size); intake fetch downloads only that folder; convert start --subfolder converts from the cached folder with any backend's converter. Laya's multilingual (mmBERT-base) and typed-decisions checkpoints convert this way.

  • The convert receipt schema allows backend (written by optional-backend conversions) and source.subfolder; tests validate real receipts against it.

  • Ports can declare the bit widths their conversions keep accurate (port_bits); intake resolve reports them as q_bits and convert start refuses others. Laya converts at 8 bits only: at 4 bits the multilingual checkpoint changes 3 of 25 reference answers.

  • intake resolve reports download_bytes, what a snapshot fetch takes; intake fetch --model-type downloads only a ported type's files (Laya: 846 MB of the 2.4 GB repo).

  • Add convert decide: answer typed questions (choice, score, noul) about a state with a converted classification model through its backend (read-only).

  • Task labels: text-classification and zero-shot-classification repos, and models a classification backend implements, are classification (use cases: moderation, routing, classification).

  • Add convert transcribe: transcribe one audio file with a converted speech-to-text model through its backend (read-only).

  • watch diff no longer reports a gated change when either side is unknown or missing.

  • intake resolve reports estimated_output_bytes (4- and 8-bit) for convertible repos from the safetensors headers; ports declare what they quantize (port_quantize). Edge0/Audio8-ASR-Infinite: 3.75 GB at 4 bits, matching the real conversion.

  • Add backend model ports: mlx-agent ships an MLX implementation of Edge0/Audio8-ASR-Infinite (audio8_asr_infinite: Voxtral Realtime audio tower, Qwen2 decoder with delay conditioning, projector, frame-length embedding, semantic VAD heads, streaming greedy decode with the 30 s rolling window) for the mlx-audio backend. intake resolve now reports Audio8 as convertible through mlx-audio; convert start --backend mlx-audio installs the port into the backend before converting. Conversion quantizes the decoder only.

  • Task labels: a model is vision-language only with a vision config or a vision module (from the mlx-vlm registry's vision files); GGUF main weights are never vision, and GGUF multimodal projectors (clip/mmproj) are labelled other.

  • Add local-path serving: serve start --path DIR serves a local model
    directory (e.g. a converted output outside the Hugging Face cache) through
    the same preview → confirm → receipt flow as --repo. The two are mutually
    exclusive; path plans are gated on the directory existing instead of cache
    membership, receipts carry path alongside repo, and status/stop argv
    matching accepts either.

  • Add intake resolve: classify a pasted Hugging Face link and report which declared MLX backend (mlx-lm, mlx-vlm, mlx-audio) converts it, or which components have no MLX implementation, without downloading weights.

  • convert scan items carry a task label (type, use cases, source, confidence).

  • Add backend list|install|remove: optional mlx-vlm and mlx-audio converters in isolated, hash-locked environments (confirmation-gated; removal moves to the Trash).

  • Add convert start --backend for repo conversions through an installed optional backend.

  • Add intake fetch and intake status: confirmation-gated, receipt-tracked downloads.

  • intake fetch skips weight formats MLX converters never read (TensorFlow, Flax, ONNX, Core ML, GGUF) in snapshot downloads.

  • Add intake port-analysis: deterministic component, weight-prefix, and custom-code analysis for architectures no backend implements.

  • Add intake port-plan: draft a porting plan from that analysis with a locally served model (loopback endpoints only).

  • Fix: convert start --repo now passes --quantize; repo conversions were previously written unquantized.

mlx-agent-docs-v0.6.0.tar.gz

sha256 b98280508791e79d8a760b8a73c7def9522c31ff4490eb77b737be4154d8b0f9