Skip to content

AbstractCore v2.19.0

Choose a tag to compare

@github-actions github-actions released this 29 Sep 05:30
· 401 commits to main since this release

AbstractCore has three install settings: pip install abstractcore (light) runs every remote
provider out of the box, pip install "abstractcore[apple]" adds every local engine an Apple
silicon Mac can run, and pip install "abstractcore[gpu]" adds every local engine an NVIDIA or
AMD machine can run. abstractcore models recommendations shows the recommended model for every
capability on every kind of machine. On Apple silicon the fit estimate matches the measured GPU
memory limit, a text model that runs only with a small context says so with the sysctl command
for more, the image route is memory-gated, and the Wan2.2 video figures are
AbstractVision/mlx-gen's measurements at AbstractVision 0.3.31's default 832x480 canvas. The
terminal console abstractcore-console 0.4.1 switches screens with the Left/Right arrows.

Install settings

  • pip install abstractcore now installs everything a remote-inference application needs: the
    OpenAI and Anthropic SDKs (OpenRouter, Portkey, Ollama, LM Studio, vLLM servers and any
    OpenAI-compatible endpoint already used the built-in HTTP client), the built-in tools, media
    inputs (images, PDFs, Office documents), token counting, Glyph compression, the HTTP server and
    console, and the AbstractVoice, AbstractVision, AbstractMusic and Abstract3D capability plugins
    with their remote backends (music and 3D on Python 3.10+). It includes no local engine. The
    install is larger than before (about 600 MB in a fresh Python 3.12 environment, mostly the
    Office-document parsers).
  • abstractcore[apple] is light plus MLX (text and image input), HuggingFace/GGUF, local
    embeddings, local voice, image, video and music engines, and Playwright for browser_probe.
    It installs everything the previous Apple aggregate installed, plus Playwright and Abstract3D.
  • abstractcore[gpu] is light plus vLLM, HuggingFace/GGUF, local embeddings, local voice, image,
    video and music engines, and Playwright. It installs everything the previous GPU aggregate
    installed, plus Playwright and Abstract3D.
  • Install hints in errors, the console and the CLI name only these three settings; the provider
    registry's installation_extras is None for every provider that talks to a server (vLLM
    included), apple for MLX, and the host's setting for HuggingFace. The Engines screen installs the HuggingFace
    engine through the host's setting (Apple silicon: apple; Linux: gpu; Intel Macs and
    Windows: the transformers stack directly).
  • The server image (ghcr.io/lpalbou/abstractcore-server) installs the light setting.
  • Migration: every previous extra name still installs, with at least the packages it installed
    before, as a deprecated alias kept for compatibility. Replace it with light (no extra), apple
    or gpu; the mapping is in Installation.

Added

  • abstractcore models recommendations [--host] [--json | --markdown] [--output PATH]: the
    recommended model for every capability (text, image input, speech output, speech input, image,
    video, music) on every machine class (Apple silicon by unified memory, Linux or Windows with an
    NVIDIA GPU, processor-only Linux or Windows, Intel Macs), or for this machine with --host.
    Each cell is recommended, covered (image input is read by the text model) or unavailable
    with the reason and what to use instead, and carries the engine, device, download size, memory
    need, fit verdict and, where needed, the GPU memory sysctl. The JSON is
    model_recommendations_v1; Python: abstractcore.config.recommendations.recommended_models()
    and recommendation_matrix(). See Recommended Models, whose
    tables are generated from the same data (scripts/update_recommended_models_doc.py --check).
  • Recommendations for speech input (Whisper base on AbstractVoice's faster-whisper engine) and
    music (ACE-Step 1.5 XL turbo on AbstractMusic's acestep backend, memory-gated like video).
    They are listed everywhere recommendations are shown and are not written by the fresh-install
    defaults, apply-recommended or models download --recommended: set them with
    abstractcore config set-default. Both models are in the download catalog with their verified
    sizes.

Changed

  • Apple silicon fit estimates follow the operator's measurement on a 24 GB Mac mini (GPU memory
    limit 17.8 GB = 16.6 GiB; Qwen3.8 27B 4-bit, 16.2 GB of weights, runs out of the box with a small
    context, and with about 30k tokens at iogpu.wired_limit_mb=20480). A text model whose weights
    fit under the GPU memory limit itself but not 8k tokens of context is tight with
    small_context: true ("Tight: it runs with a small context by default; close other apps
    first"), with no estimated token count; the 8 GB Mac's Qwen3.5 9B and the 24 GB Mac's Qwen3.8 27B
    read so. The only GPU memory limit ever suggested is the highest safe one, RAM minus
    max(4 GiB, 12.5% of RAM) (was the smallest whole GiB that fits, under RAM minus 8 GiB): 20480 MB
    on 24 GB, 114688 MB on 128 GB (the Flash-Next command was 117760); a tight Apple silicon fit
    carries it as raised_limit, and the one measured context is stated with it. MLX's reserve
    beside the weights and cache is a flat 2 GiB (was max(2 GiB, 5%)). The sentences name the GPU
    memory limit itself, never only the remainder as "what this computer can give a model", and no
    longer say "a smaller model from the catalog is the safe choice". models recommendations
    entries gain context: {small, measured}, and their gpu_limit_command also carries the
    command for a tight text fit.

  • The Apple silicon text tiers are unchanged from 2.18.0 (operator ruling): below 24 GiB Qwen3.5
    9B (an 8 GB Mac keeps it, read as tight with a small context), 24 to below 128 GiB Qwen3.8 27B,
    128 GiB and above Qwen3.8 Flash-Next. A text tier whose model has no MTP build would carry no
    MTP speculation policy.

  • The recommended image route (FLUX.2 klein 4B, about 8.5 GiB) is memory-gated like video and
    music: an 8 GB Mac gets no image route, with the reason.

  • Wan2.2 video memory needs are AbstractVision/mlx-gen's peaks measured at each model's default
    canvas in AbstractVision 0.3.31 (832x480; tiled VAE decode): TI2V-5B about 16.6 GiB (832x480,
    121 frames; image-to-video, the larger of its two tasks -- text-to-video peaks at 16.3 GiB, same
    MLX allocator measurement; the one checkpoint serves both, so the route uses 16.6), so the recommended video route is written on Macs with 32 GiB of unified memory or
    more, and a 24 GiB Mac is told the GPU memory limit that makes it fit; T2V-A14B and I2V-A14B
    about 38.3 and 38.4 GiB (832x480, 81 frames). Every sentence labels the figure as that engine's
    (it keeps the text encoder and VAE in memory), not the model's own requirement: catalog
    resident.measured_with. Other measured sizes stay in the catalog: resident.larger_canvases
    (new; TI2V-5B at 1280x704: 25.4 GiB; T2V-A14B at 1280x720: 53.5 GiB; I2V-A14B at 1280x720: 71.7 GiB, measured with AbstractVision 0.3.30's untiled decode) and resident.smaller_canvases
    (T2V-A14B at 640x352: 35.1 GiB). Catalog artifacts carry larger_canvas: {canvas, need_bytes, verdict} (with
    a fit note) when a measured larger size also fits the machine, and smaller_canvas when only a
    measured smaller size does.

  • The AbstractVision floor is abstractvision>=0.3.31 (light, apple, gpu):
    the memory figures above hold for its 832x480 defaults and tiled decode, and an older
    AbstractVision would generate TI2V-5B at 1280x704 by default (about 60 GiB).

  • capability_defaults.RECOMMENDED_MODELS is the one table of recommended models;
    RECOMMENDED_CAPABILITY_DEFAULT_ROUTES and RECOMMENDED_MODEL_DOWNLOADS are its starter-set
    views and keep their values.

  • The reason a memory-gated recommendation is unavailable says whether the memory need is
    measured or estimated.

  • Terminal console (abstractcore-console 0.4.1): Left/Right switch to the previous and next
    screen in browse mode, wrapping at both ends, like Ctrl+P / Ctrl+N; a focused text field,
    screen bar or dialog keeps the arrows. See console-tui/CHANGELOG.md.

Fixed

  • A saved image or video route whose catalog model does not fit this computer's memory is flagged
    route_unavailable (grid, abstractcore config defaults, apply-recommended) with the reason
    the recommendation gives, from the same fit verdict: an 8 GB Mac's saved FLUX.2 klein 4B route
    no longer reads as fine and runs out of memory at first use. A tight fit, or one that needs
    the GPU memory sysctl, is not flagged, nor is a video route that runs at a measured smaller
    canvas (the route the reason tells a 64 GB Mac to set); the check never changes the saved routes.
  • abstractcore config apply-recommended --force can now clear a saved image or video route whose
    model does not fit this Mac, where nothing recommended runs either.
  • input.image says why it has no recommendation where the recommended text model does not read
    images or no text engine runs (recommended_unavailable_routes, the grid's
    recommendation_unavailable), instead of a bare "not configured".
  • A measured memory need (the Wan2.2 video resident peaks) is the need itself: the fit no longer
    adds the estimate's overhead (max(0.5 GiB, 5%)) on top, which made TI2V-5B read "needs about
    17.4 GiB ... (measured ...)" for its measured 16.6 GiB (estimate_fit(need_measured=True)).
    T2V-A14B and I2V-A14B (about 38.3 and 38.4 GiB) now fit a 48 GiB Mac once its GPU memory limit
    is raised (needs_gpu_limit, with the command) instead of reading too_large.
  • The light install no longer imports unstructured to learn whether Office documents are
    supported (importlib.util.find_spec): building a media handler, for a text file say, sent
    unstructured's telemetry request to packages.unstructured.io and ran nvidia-smi. Processing an
    Office document imports it with its telemetry opted out (SCARF_NO_ANALYTICS and
    DO_NOT_TRACK default to true; an explicit value is kept).
  • Install hints name the three settings: the HuggingFace provider on an Intel Mac or Windows
    installs the transformers stack directly (as the Engines screen does), never abstractcore[gpu]
    (vLLM); missing MLX points to abstractcore[apple]; missing llama.cpp / Transformers to this
    machine's setting; a missing OpenAI or Anthropic SDK (shipped with every install) to
    pip install -U abstractcore.