AbstractCore v2.13.41

2.13.39 and 2.13.40 were not published to PyPI separately; their changes (listed below) ship
with 2.13.41.

Added

• Native MLX runtime. In-process Qwen3.8-27B and Qwen3.8-Flash-Next support on the mlx
  provider with embedded multi-token prediction (MTP), per-request draft depth, bounded native
  prefix caching (RAM and optional SSD), image input with MTP on or off, and speculation
  telemetry. Opt-in request scheduling (mlx_batching=True) adds continuous batching, bounded
  admission and shared model ownership; the AbstractCore server and single-model endpoints admit
  scheduled requests concurrently. See native MLX runtime,
  speculative decoding and
  M5 Max benchmarks.
• Core-owned MTP defaults and execution-capability discovery. Fresh configurations use draft
  depth 2 on compatible native-MLX models with cached heads; existing settings are preserved.
  Applications keep per-request Off/depth overrides, and unsupported strict requests fail
  explicitly. Defaults never download weights.
• Generation cancel. Pass a threading.Event as cancel_event= to generate(): providers
  that declare supports_generation_cancel() stop the running decode (MLX per sampled token and
  during mlx-lm prefill; HuggingFace transformers and llama.cpp per step or prefill slice;
  OpenAI-compatible, LM Studio, vLLM, OpenRouter, Portkey and Ollama by severing the in-flight
  HTTP request). A stopped call raises the new typed GenerationCancelledError, which is never
  retried. See generation cancel.
• Safe eject and reload. unload_model() cancels and drains the instance's in-flight calls
  first (bounded, 30 s by default); MLX and HuggingFace refuse to free memory under a running
  decode. MLX and HuggingFace gain load_model() and reload on demand after an eject. HTTP
  providers stay usable after unload_model().
• Live generation phase and prefill progress. generate(progress_callback=...) (alias
  on_progress=) receives JSON-safe {"kind": "llm", "phase": "prefill" | "generate" | "complete", ...} events with prompt, cached and generated token counts, prefill position and
  throughput, TTFT and tokens per second, at most one event per 0.5 s
  (ABSTRACTCORE_PROGRESS_MIN_INTERVAL_S). Implemented on MLX and HuggingFace; providers report
  support through supports_text_progress_events() and HTTP providers emit nothing.
• AbstractCore server: a client disconnect cancels HTTP-backed providers as well as the MLX
  scheduler, and a cancelled request answers 499.
• Host memory and residency visibility (2.13.40). get_memory_snapshot(), host-wide
  sweep_loaded_models(), list_loaded_models() on every provider, per-model memory on
  residency records, prompt-cache size stats, POST /acore/prompt_cache/key_meta, and model
  locks (POST /acore/models/lock / unlock).
• Reasoning effort on local providers (2.13.39). thinking="low|medium|high|xhigh" is
  enforced where the model supports it, with a reasoning_effort compatibility net for
  OpenAI-compatible servers. A reasoning effort configured on the text route applies to calls that
  name none, and is settable through the server.
• Web and document tools. fetch_url returns one canonical structure-preserving markdown
  content, can render JavaScript pages, escalates bot-challenge 403s to a real browser, reads
  Reddit threads through their feeds, recovers headlines from site feeds when every client is
  refused, and reports honest terminal classes (captcha_required, paywall, login_required,
  blocked_by_site). browser_probe verifies rendered pages. An SSRF guard validates every hop.
  See web and document tools.
• analyze_media tool gives text-only agents delegated sight through a vision model.
• analyze_code covers 34 languages, marks every place output is withheld, and ends with
  runnable recovery steps.
• Persistent shell sessions (tools/shell_session.py, tools/shell_tools.py), schema-aware
  tool-argument coercion, tool risk tiers and side-effect tags, and an authoritative builtin-tool
  inventory (abstractcore.tools.inventory).
• Retry wall-clock budget (RetryConfig.max_total_wall_clock_s,
  create_llm(..., retry_wall_clock_budget_s=...)) and an LLM read-idle timeout that aborts a
  stalled stream.
• Provider endpoint profiles registered at runtime (register_runtime_provider_profile) and
  create_llm(..., prompt_cache_key=...).
• Durable prompt-cache artifacts are gated by engine, tokenizer, model-config and weights
  identity, and support q8 KV storage. MLX warm-cache reuse covers Gated-DeltaNet hybrids
  (Qwen3.5 / Qwen3.6 / Ornith 1.0).
• Machine-level data-home registry (abstractcore.utils.data_registry).
• Model registry: Anthropic generation-5 models, Poolside Laguna, Nemotron 3 Nano 4B and
  recent Hugging Face models.

Changed

• Dependencies. MLX and Apple installation profiles require mlx>=0.32.2, mlx-lm>=0.31.3
  and mlx-vlm>=0.7.1,<0.8.0; mlx-vlm is part of every MLX profile, so mlx-vision is a
  compatibility alias. Media plugin floors: abstractvoice>=0.11.2, abstractvision>=0.3.29,
  abstractmusic>=0.1.15. The embeddings, all and full-dev extras accept NumPy 2 on every
  Python version, so they combine with the MLX profiles (mlx-vlm 0.7 requires NumPy 2).
• Configuration is config-first. A config-set API key supersedes the environment variable;
  vision, voice and music behavior on the standalone server is read from configuration. Writing
  one field of a capability route keeps the rest of the route.
• Native MLX num_draft_tokens counts proposals excluding the target seed, including
  separate 27B MTP heads.
• Native MLX prefix-cache checkpoints default to a 256-token interval with 4 resident
  snapshots; the APC_CHECKPOINT_* environment variables still override them.
• skim_url uses the same extraction and SSRF guard as fetch_url; time_range and
  require_in accept plain-English values.
• Unload also frees session caches on in-process providers, and pinned on managed
  residency records is an alias of locked.

Fixed

• MLX stream=True no longer raises TypeError on the mlx-lm / mlx-vlm lanes.
• response_model= keeps attached media on MLX; streaming responses carry media_delivered and
  media_dropped; images with uncommon extensions are recognized by content.
• MLX responses report finish_reason="length" at the output cap; vision prompt usage includes
  expanded image tokens.
• Prepared prompt-cache prefixes accept thinking= and use the same reasoning control as
  generation; MLX prepared-prefix forks report a visible #FALLBACK when the context diverges.
• Structured output on strict-schema backends falls back to prompted generation; MLX and
  HuggingFace Outlines output is validated.
• Tool-call history with non-string argument values renders on LM Studio and HuggingFace GGUF.
• Anthropic and OpenAI native providers keep role: "system" messages; Anthropic cache usage is
  reported; streamed OpenAI-compatible usage is accounted.
• MCP tool names survive strict native endpoints; Ollama num_ctx is forwarded.
• execute_command timeouts kill the whole process tree and return captured output.
• edit_file preserves CRLF line endings and can write literal escape sequences.
• Configuration loading never silently regenerates defaults.
• GGUF residency reports weights across split shards; device.host_in_use_bytes measures
  accelerator-heap memory.