Repository navigation
AbstractCore v2.19.1
This release makes the gpu setting work on Linux + NVIDIA and lets abstractcore[gpu] install on
Windows x86_64 with wheels only. Run on a Quadro RTX 5000 (driver 595, CUDA 13): LM Studio text,
llama.cpp GGUF on CUDA, faster-whisper on CUDA, Supertonic speech and Diffusers FLUX.2 klein
images. vLLM installs (0.22, PyTorch 2.11) but was not validated: its first start needs a C
compiler (build-essential). The Windows changes are not yet validated on Windows hardware.
Dependencies
openai>=1.0.0,<3.0.0(was<2.0.0). The old cap held vLLM at 0.11.0 inabstractcore[gpu],
which does not start with Transformers 5; on Linux the gpu setting now resolves a current vLLM
(0.22 with torch 2.11 on CUDA 13 at release time). The OpenAI API calls AbstractCore makes are unchanged in openai 2.x.- Floors:
abstractvision>=0.3.32(itsgpusettings no longer install MLX-Gen, about 2.1 GB less
on Linux, and leave stable-diffusion.cpp out on Windows) andabstractvoice>=0.13.1
(faster-whisper on CUDA with CUDA 12 cuBLAS; speech-to-text without TTS credentials), in the
light install and in theappleandgpusettings. abstractcore[gpu]installs on Windows x86_64 with wheels only:
vllm; sys_platform == 'linux'(vLLM is Linux only upstream) and
llama-cpp-python; sys_platform != 'win32'(a source build on PyPI; the AbstractFramework
installer adds llama.cpp's prebuilt CUDA / Vulkan / CPU build on Windows). The deprecated
all-gpuandvllmaliases still map togpu.
Changed
- Install hints name only the three settings. Every error, CLI message, engine install plan and
doc sayspip install -U abstractcorefor a light dependency,abstractcore[apple]or
abstractcore[gpu]for a local engine, and, on a host with no local-engine setting (Intel Mac,
Windows on ARM), that the capability is not available there. No message advises a bare package
or a plugin's own extra. - The Engines screen installs
abstractcore[apple]/abstractcore[gpu]for MLX, llama.cpp, vLLM
and Hugging Face.abstractcore --download-vision-modelno longer pip-installs
transformers/torch on its own. - Windows x86_64 has the
gpusetting in install hints and the Engines screen. The llama.cpp hint
and engine row there say that the AbstractFramework installer adds llama.cpp's prebuilt GPU
build and that, with plain pip, llama.cpp is not part ofabstractcore[gpu]on Windows. - The camera endpoints and capability hint say the camera plugin (
abstractcamera) is outside the
install settings instead of printing a pip command. - Recommendations on NVIDIA GPUs:
output.imageisdiffusers/
black-forest-labs/FLUX.2-klein-4B(AbstractVision's Diffusers backend on CUDA, float16).
With AbstractVision 0.3.32's automatic model CPU offload it generates 768x768 in about 17 s with
an 8.3 GiB GPU peak on a 16 GB card (about 15 GiB of system RAM holds the idle weights). It is
fit-gated with that measured peak: an 8 GB card is reported unavailable with a GPU-memory next
step. Apple silicon keeps MLX-Gen; hosts without a GPU stay "not available". The catalog row is
now named "FLUX.2 [klein] 4B". - The recommended LM Studio text download is
qwen/qwen3.5-9b@q4_k_m(was@4bit, an MLX variant
that LM Studio does not offer off Apple silicon, where LM Studio is the recommendation). apply-recommended, the defaults grid andmodels download --recommendedread the full host
probe off Apple silicon, so an NVIDIA host gets its image route fromapply-recommended.
Fixed
- Speech-to-text runs on the configured
input.voiceroute. A transcription
(output={"modality": "text", "task": "transcription"}) with no provider used AbstractVoice's
default OpenAI engine instead of the configured one (for example local faster-whisper). The
route now fills a missing provider/model/base_url under the same rules as every other route (a
provider named on the request, or a call-levelstt_provider/stt_model, still wins), also for
audio given as an artifact reference without a content type. - Linux + NVIDIA: llama.cpp's CUDA wheels load without importing torch first.
prepare_llama_cpp_import()also runs on Linux and, for a CUDA build of llama-cpp-python,
preloads the NVIDIA wheels'libcudart/libcublasLt/libcublas(CUDA 12 and CUDA 13
layouts), so GGUF models run on the GPU instead of the CPU. - GGUF on CUDA: when llama.cpp cannot allocate a large context on the GPU, the load takes the next
smaller context on the GPU and falls back to the CPU only after the smallest one fails there too
(a CPU retry at a 262144-token context could exhaust system memory). Apple silicon keeps its
immediate CPU retry. - Windows: before AbstractCore first imports
llama_cpp, it adds PyTorch'storch\libfolder
(cuBLAS, cudart) to the DLL search (os.add_dll_directoryandPATH), so llama.cpp's prebuilt
CUDA wheel can find those libraries without the CUDA toolkit (not yet validated on Windows
hardware)
(abstractcore.utils.windows_dll.prepare_llama_cpp_import; torch is not imported). - Windows: the GGUF lane runs
llama_backend_init()before asking llama.cpp whether it can offload
to the GPU (the Windows CUDA / Vulkan wheels ship the GPU backend as a separate library loaded at init; not
yet validated on Windows hardware). - The server's
/v1/responsesusage reportsinput_tokens_details.cache_write_tokens(0 when the
provider reports none), which the Responses usage schema of openai 2.x requires.