Skip to content

0.8.0 — thirteen catalog models; hybrid chats resume from a checkpoint; four models stop at the end of their KV cache

Latest

Choose a tag to compare

@john-rocky john-rocky released this 14 Nov 12:04
· 2 commits to main since this release

Thirteen catalog models join (67 → 80): six decision models on five new kinds, zero-shot text classification, LFM3 1.7B with tool calling, MiniCPM-V 4.7 1B, Fun-ASR-Nano, Audio8-TTS, 8-speaker diarization and quantile forecasting. Chats on a hybrid model resume from a checkpoint instead of prefilling the whole conversation again. VibeVoice, VoxCPM, VoxCPM2 and Unlimited-OCR stop at the end of their KV cache, and a graph compiled for one iPhone is picked only on that iPhone. This is a minor release: some code written for 0.7.3 needs an edit, listed under Changed. A package that depends on from: "0.7.3" gets 0.8.0 the next time its packages are updated (swift package update, or Update to Latest Package Versions in Xcode); one that depends on exact: "0.7.3" stays on 0.7.3 until that line changes. Built and gated on macOS 27.0 (26A428) and Xcode 27 (27A266a).

Start here: installation and smallest working example · Examples/TextClassify · Examples/Week

The systemone command-line tool, from Homebrew:

brew upgrade systemone                                        # from 0.7.3

The Swift package, in Package.swift:

.package(url: "https://github.com/john-rocky/coreai-kit", exact: "0.8.0")

Changed

  • What code written for 0.7.3 may need.
    • VibeVoicePaths.inBundleDir(_:glueDir:voicesDir:embedTokens:) now throws when the folder holds
      only graphs compiled for other devices (under Fixed): call it with try.
    • A switch without default over CatalogEntry.Kind, Decision.Format, VisionError,
      KitBundleError, VibeVoiceError or VoxCPMError: each has new cases (the kinds and
      jointHead under Added, the errors under Fixed). Add a default.
    • A reference to one of the five functions below by its 0.7.3 full name, without a call (for
      example let make = SpeakerSegment.init(speaker:startFrame:endFrame:)), no longer compiles: each
      gained parameters with default values, so its full name changed. Wrap it in a closure instead
      (let make = { SpeakerSegment(speaker: $0, startFrame: $1, endFrame: $2) }). A direct call that
      leaves the new parameters out builds as before.
      • SpeakerSegment.init(speaker:startFrame:endFrame:) gained frameSec.
      • KitDiarizer.segments(from:bridgeFrames:) gained frameSec.
      • CatalogEntry.init(…) gained assets.
      • CatalogEntry.Variant.init(path:sizeMB:) gained engine and architectures.
      • VLArchitecture.init(…) gained ropeShiftFrameTail and assistantPrefix.

Added

  • Chats on a hybrid model resume from a checkpoint — on Qwen3.5, LFM2.5, LFM3 and Granite 4 the
    recurrent state cannot be cut back, so every ChatSession turn and every KitLanguageModel call used to
    re-prefill the whole conversation. Both now ask the engine to checkpoint the state at the end of the prompt's
    last finished message (InferenceOptions.checkpointAfter, just past the last turn-end token such as
    <|im_end|>; the engine saves it while it prefills, without a call of its own, on the chunk grid of a
    chunked prefill). The next turn rewinds to the checkpoint, or keeps everything when it extends the previous
    prompt, and prefills only what follows. On an M4 Max (2026-10-30), turn 2 of a conversation about a
    1,584-token document on LFM3-1.7B: 837.8 ms to the first token with the history re-prefilled, 38.6 ms from
    the checkpoint; replies identical in both ways on LFM3-1.7B, Qwen3.5-0.8B and LFM2.5-1.2B. On an iPhone 17 Pro,
    Qwen3.5-0.8B: 19.8 → 3.2 s on a 484-token conversation, replies identical. A first turn costs nothing
    measurable. A template that rewrites the earlier history, and the sequential engine, re-prefill as before.

  • coreai-models pinned to 0.2.9-zoo (was 0.2.8-zoo) for the checkpoint above. It also brings the padded
    last chunk: a bundle whose metadata declares that its prefill function takes a padded chunk
    ("prefill_pad_tail": true) feeds what is left of a prompt after its full chunks as one call instead of up
    to 63 single steps (LFM3-1.7B, 1,000-token prompt: 0.692 → 0.500 s to the first token on an M4 Max). All
    lockfiles follow.

  • Tool calling for LFM3 — KitLanguageModel declares .toolCalling for a bundle whose vocab carries LFM3's
    <function= / <parameter= tokens and renders the tool turns the way LFM3's chat template does (<tools> lines
    in json.dumps form, XML <tool_call> calls, <tool_response> results, reasoning replayed inside the tool loop),
    then parses the model's XML calls with values typed by the tool's schema. The render equals transformers'
    apply_chat_template string and ids on 30 conversations; LFM2.5 bundles still take no tools.

  • Streams that start inside a thinking block — when a rendered prompt ends with an open thinking marker
    (LFM3's template with thinking on ends <think>), ChatSession and KitLanguageModel parse the reply as
    thinking until the close marker, so the reasoning no longer reaches the answer text. A prompt that ends anywhere
    else parses as before.

  • LFM3 1.7B — catalog id lfm3-1.7b (kind: chat, macOS), the zoo's int8 bundle on the pipelined engine: it
    thinks by default and calls tools through LFM3ToolDialect.

  • An engine hint per platform — a catalog variant may name its own engine (CatalogEntry.Variant.engine), which
    wins over the entry's on that platform (CatalogEntry.resolvedEngine); ChatSession(catalog:), TypedDecisions and
    KitTextNormalizer load with it. lfm3-1.7b gains an iPhone variant on the static-shape engine: the Neural Engine
    form, compiled per iPhone generation into the repo's ios-<arch>/ (ios-h19p/ for the iPhone 18 Pro at this upload;
    no ios/). A kit without the key reads the entry's engine ("pipelined", the Mac bundle's).

  • Which iPhones have a bundle, before the download — an ios variant in a repo with no ios/ lists the
    architectures it has an ios-<arch>/ for (CatalogEntry.Variant.architectures; lfm3-1.7b lists h19p).
    CatalogEntry.unavailableReason reads it, and the architecture of a path inside an ios-<arch>/ folder, against
    the device's (AIModel.deviceArchitectureName): nil when the phone has a bundle, the Mac, or a phone Core AI does
    not name, else a sentence for a picker. ModelCatalog.entry(forID:), which every catalog: initializer rides,
    throws unsupportedDevice with it before any download. A kit without the key ignores it.

  • ChatSession.Configuration.enableThinking — renders enable_thinking into the bundle's chat template
    (nil, the default, leaves the template's own default); false gives LFM3 and the Qwen3 family their thinking-off
    prompt, so the response budget goes to the answer. On a template that does not read the variable (LFM2.5 2.6B,
    Nanbeige4.1 3B), false closes an empty think block after the generation prompt instead.

  • MiniCPM-V 4.7 1B — catalog id minicpm-v-4.7-1b (kind: vlm), loaded by KitVisionModel through the new
    VLModelID.miniCPMV47 / VLArchitecture.miniCPMV47: OpenBMB's 4.6 vision tower (448 px, 64 image tokens) in front
    of a decoder that places the image on a canvas (M-RoPE over T, H and W) with an untied head, answering with
    thinking off. The prompt is the reference processor's — <image_id>0</image_id><image>, the 64 pads, </image> —
    and two new VLArchitecture fields, both defaulting to the old behaviour, carry the rest: ropeShiftFrameTail
    (</image> and the newline after it sit on the canvas, so the rope shift starts after them) and assistantPrefix
    (the empty think block the chat template writes at the start of every assistant turn). The int8hu decoder and the
    int8lin tower are the pair the zoo gated on an iPhone 18 Pro (iOS 27.2): 98.0 tok/s decode and 68.2 ms per image
    in its bench.

  • Timestral-209M — catalog id timestral-209m (the new kind: quantileForecasting), driven by the new
    KitTimestralForecaster and the op CoreAI.forecastQuantiles(_:horizon:options:): a number series in, horizon
    steps of 21 quantiles (0.01 to 0.99) out, as a QuantileForecast with median and quantile(_:). By default
    each graph runs where its form computes right: the form that is fp16 throughout (fp16p7-qkv4-tanh-conv) on the
    Neural Engine, the other forms on the GPU. Asking for any other unit for the fp16-throughout form throws
    KitTimestralForecasterError.wrongComputeUnits, and an output at fp16's limit throws .saturatedOutput, instead
    of returning a forecast. Download: 1,214 MB on macOS and on iOS; license LicenseRef-Timestral-Modified-MIT. The
    kind is new: a kit built before it decodes the entry as unknown and leaves it out of available(_:). Examples/Forecast runs TimesFM
    2.5 or Timestral-209M on pasted numbers or a CSV file for a horizon you choose, and its CLI takes the same inputs.

  • Examples/TextClassify — your own messages, sorted into categories you name. Paste them (one a line, or blocks
    separated by a blank line), import a .txt, .md or .csv file (UTF-8; anything else is refused with the reason), or
    take the sample inbox; at most 2,000 messages. Name 2 to 12 categories in place of the eight support intents.
    Every message still gets an urgency and a sentiment from the same forward, and By category lists each category
    with its most urgent messages first. On a Mac, the download, the load and a sort hold a user-initiated activity,
    so App Nap no longer slows a download in a background window to 0.1–0.2 MB/s. Measured 2026-10-10: 100 pasted
    messages at 30.0 to 31.0 ms median each on an M4 Max and 38.4 to 38.6 ms on an iPhone 18 Pro, every message with
    the same three answers as the sample inbox.

  • Examples/Week — your own week planned on one tap: every event is one decider-0.8b decision (what it
    needs before it, seven options), and the events that need something are gathered under Before your week, in
    time order. The week comes from your calendar (this week, read only: nothing is ever written to a calendar),
    from pasted lines (Mon 11:00 Title · Place · Notes: …, or a week-cli dump --out JSON file; unreadable
    lines are counted and skipped, at most 200 events), or from a seeded sample week of fictional events. Add
    reminders asks for Reminders access only when pressed. week-cli run gives the same loop and JSON on a Mac
    and gives the app's 20 answers back, event by event. Measured on an M4 Max (macOS 27.0, 2026-10-09): 20
    events in 12.90 to 13.01 s, 627 to 651 ms median per event.

  • d1-3B — catalog id d1-3b (the new kind: tokenDecision, format: optionRows), driven by the new KitD1Decider
    (Sources/CoreAIKit/D1/): Liquid AI's decision model from LFM2.5-VL-3B (LFM Open License v1.0; a SigLIP2 vision tower
    and the LFM2 hybrid decoder). A state (text or JSON), pictures and typed questions (noul, choice, score) go in; every
    option of every question comes back with a probability; nothing is generated. Each question is its own row in the
    provider's chat form; the decoder (no vocabulary head, a static 64 tokens a call from zeroed states) returns the hidden
    state at every position, and the host reads the row's last one against the tied embedding rows of each option's tokens
    (head/option_rows, 2,134 ids) in float64, each option's highest logit, a softmax over the options. A picture is
    capped at one megapixel, cut into the processor's crops and read by the vision tower once per crop. The host is the
    model zoo's apps/D1 library (e36ad15) with every numeric path unchanged: across its eleven files only names
    (prefixed D13B; the zoo's D1Decider is D13BPipeline), access, @available(macOS 27, iOS 27, *), the Intel-Mac
    guard on the three files that hold Float16 and a provenance line differ, and undoing those gives the zoo's files back
    byte for byte. Its readout's products go through the BLAS calls NumPy makes, in a new C target, CoreAIKitD1BLAS
    (defines only, no unsafe flags; the zoo's D1BLAS with its function named coreaikit_d13b_matvec). Two decoders,
    one contract: 5,598 MB for the Mac (fp16) and 3,740 MB for the iPhone (MLP linears int8 per block of 32); the vision
    tower (854 MB), named in a new assets.tower field that a kit built before it ignores, downloads when a request
    carries a picture. An iPhone app needs com.apple.developer.kernel.increased-memory-limit (without it the
    decoder does not load; the kit cannot add it). A row holds at most 4,032 tokens and a request's pictures at most 2,816
    image tokens, refused before any graph call in the provider's words. The kind is new: kits 0.1.0–0.7.3 decode an
    unknown kind as unknown and leave the entry out of available(_:); the bundle's metadata says kind: decision-backbone with no decision.head, which TypedDecisions cannot read, and TypedDecisions refuses it by its
    kind before downloading anything (so CoreAI.decide and CoreAI.systemOne do too). systemone serve, systemone mcp and SystemOne.backend(catalog:) load KitD1Decider for the kind (configuration.sharePrefix reaches it), and so
    do decide-cli ask (--image) and serve (--bundle <decoder> [--tower <tower>]). prepare(state:) and
    decide(prepared:questionsJSON:) answer later questions on a kept state; readout(requestJSON:images:shared:) gives
    each row's ids, hidden digest, logits and p and the provider's response. D13BTests (the catalog entry and the
    assets.tower field, the refusals before a download, the request in the provider's form, the option order, 24 rows'
    text and readout against the Python reference without files; with a tokenizer, the zoo's 393 fixture rows against
    host.py; with the bundles, 28 questions — 16 records of text and 2 pictures — bit for bit against the Python
    reference's run of the same bundle, the shared prefix, a prepared state, the wire and the five refusals, measured on an
    M4 Max: 28 of 28 rows and 18 of 18 responses). On the wire, instructions, levels and descriptions given as JSON values
    read as their JSON text, the kit's rule for every model (the provider's code takes text only;
    readout(requestJSON:) refuses them as it does). systemone serve --model d1-3b passes conformance/check.py (22
    requests and the three routes).

  • d1-omni-600M — catalog id d1-omni-600m (the new kind: omniDecision, format: markerScores), driven by the new
    KitD1OmniDecider (Sources/CoreAIKit/D1Omni/): Liquid AI's decision model (LFM Open License v1.0; an
    LFM2.5-Encoder-350M trunk with a typed decision head, a SigLIP2 vision tower and a FastConformer audio tower). A state
    (text or JSON) with images or one 16 kHz clip, and typed questions (noul, choice, score) go in; every option of every
    question comes back with a probability; nothing is generated. Each question is its own row in the publisher's form, a
    <|mask|> marker before each option, read from one forward pass of a static fp16 decision graph at the smallest of
    seven lengths (64 to 4,096 positions) that holds the row, behind the prefix rows of the vision graph (one call per
    image crop) or the audio graph (5 / 10 / 20 / 30 s buckets). The host is the model zoo's apps/D1Omni library
    (f9e0e09) with every numeric path unchanged: across its twelve files only names (prefixed D1; the zoo's D1Omni is
    D1OmniPipeline), access, @available(macOS 27, iOS 27, *) and a provenance line differ, and a request's state is
    tokenized once for all its questions. 6,458 MB, the same .aimodels on macOS and iOS: the platform folder's twelve
    bundles download whole on first use, each its own download, so an iPhone takes the JIT ios/ graphs and never the
    repo's ios-h19p/ AOT set (which has no L4096); a graph loads the first time a row or a medium needs it. The kind is
    new: kits 0.1.0–0.7.3 decode an unknown kind as unknown and leave the entry out of available(_:) (each tag's
    ModelCatalog.swift, read); the bundle's metadata says decision.head = "encoder", so decision would put it in
    front of laya's encoder path, which cannot read its rows. TypedDecisions refuses it by its kind before downloading
    anything. /v1/systemone gains audio (one clip: a data URL, base64, or a file path, which SystemOneServer reads
    only on 127.0.0.1); a request with audio may leave state out, and every backend but KitD1OmniDecider refuses
    it with a 422 that names the model. systemone serve, systemone mcp (the decide tool's schema gains audio) and
    SystemOne.backend(catalog:) load KitD1OmniDecider for the kind, and so do decide-cli ask (--image, --audio)
    and serve (--bundle takes the repo's macos/ folder). D1OmniTests (the catalog entry, the refusals, the wire's
    audio, the request in the publisher's form, the readout of 38 fixture rows against the publisher's response text;
    with the repo's tokenizer/ and reference/, the zoo's 360 public rows token for token; with the bundles, 14
    requests — 20 text rows at 64 to 4,096 positions, 3 images, 3 clips — bit for bit against the zoo's Swift host,
    measured on an M4 Max: 38 of 38 rows and 14 of 14 responses).

  • Kev-0.8B and Kev-4B — catalog ids kev-0.8b and kev-4b (the new kind: rowDecision, format: pointerHead),
    driven by the new KitKevDecider (Sources/CoreAIKit/Kev/): Jared Palmer's decision models, a rank-16 LoRA adapter
    and a pointer head on Qwen3.5-0.8B-Base and Qwen3.5-4B-Base (Apache-2.0). A state (text or JSON) and typed questions
    (noul, choice, score) go in, and every option of every question comes back with a probability; nothing is generated.
    Each question is its own row: the state, then the question and its options between the author's delimiter tokens. The
    host is the model zoo's apps/Kev library (9e06b5a) with every numeric path unchanged: the author's rendering and row
    form, the decoder (fp16, no vocabulary head, a static 128 tokens per call, each Gated DeltaNet recurrence in an fp32
    Metal kernel) returning the hidden state at every position, and the author's pointer head on the host in float64 at
    the author's temperature, each p rounded to float32 once. The shared prefix is on by default: a request's state runs
    its whole 128-token calls once, and on this graph every row's hidden state equals the direct run's bit for bit.
    prepare(state:) and decide(prepared:questionsJSON:) answer later questions on a kept state. A choice lists up to 255
    options; a row holds at most 3,968 tokens. 1,520 MB for Kev-0.8B, the same .aimodel on macOS and iOS (the zoo gated
    it on an iPhone 18 Pro; the kit has not run on a phone), and 8,431 MB for Kev-4B, macOS only. The first load specializes
    the graph: 3.3 s for 0.8B, 16.6 s and a 15.55 GB runtime cache entry for 4B in the zoo's runs. The kind is new: kits
    0.1.0–0.7.3 decode rowDecision as unknown and leave both entries out of available(_:) (each tag's own catalog
    code, compiled and run on this catalog.json); decision would list them in a shipped kit's systemone models, and
    loading one there would download the bundle for a TypedDecisions that cannot read it. TypedDecisions refuses them by
    their kind before downloading anything. KitKevDecider refuses a bundle whose graph takes a dynamic query length
    (measured in the zoo and not shipped: its memory grows). systemone serve, systemone mcp and
    SystemOne.backend(catalog:) load KitKevDecider for the kind, and so do decide-cli ask, serve and parity.
    CoreAI.decide and CoreAI.systemOne load through TypedDecisions, so they refuse Kev by its kind.
    On an M4 Max through decide-cli parity, downloaded from the catalog pin, on the zoo fixture's 155 records that carry
    their text (186 questions): the author's row ids on every question, the argmax equal to the author's fp32 read-out on
    every question with a top-2 margin above 0.02 (Kev-0.8B 181/181, near-ties 4/5; Kev-4B 182/182, near-ties 3/4), max
    |Δp| 0.0124 and 0.0153, and the hidden rows and p bit-equal to the zoo's Swift host's run of the same bundle on all 186
    questions of each model, the answers on all 155 records (Kev-0.8B with the shared prefix on and off, Kev-4B on).
    systemone serve --model kev-0.8b passes conformance/check.py (22 requests and the three routes); a request with
    images gets a 422. KevTests (the catalog entries, the request in the author's form, the refusals; with
    KIT_KEV_GATE=1 / KIT_KEV_GATE_4B=1, 10 Kev-0.8B and 3 Kev-4B records bit for bit against the zoo's Swift host,
    direct and shared).

  • clef-flash — catalog id clef-flash (the new kind: jointDecision, format: jointHead), driven by the new
    KitClefDecider (Sources/CoreAIKit/ClefFlash/): Cloudflare's decision model from Qwen3.5-9B (Apache-2.0). Every
    question of a request (noul, choice, score) is read in one pass by its joint schema head, with the state as text or
    JSON and at most one image, and every option gets a probability; nothing is generated. The host is the model zoo's
    apps/ClefFlash library (5ef2247) with every numeric path unchanged: the request rendered as the author's
    encode_record() (canonical JSON, each piece tokenized alone), the image resized by Pillow's own integer bicubic to a
    fixed grid (g448 by default, 196 image tokens; g256, 64) and read by that grid's tower, the decoder (fp16, no
    vocabulary head, 64 tokens per call) returning the hidden state at every position, the head graph over span means, the
    last token and each option's rows of a 2.03 GB fp16 lm_head table on the host, and a float32 softmax per question at
    temperature 1. Up to 16 questions and 128 options a request. 18,178 MB: the decoder, the head and the table; a tower
    (909 MB g256, 912 MB g448) downloads the first time an image asks for its grid. macOS only (the decoder alone is
    15.9 GB); the zoo measured a first load that specializes the decoder in 56 s into 31.8 GB of the runtime's cache, and
    4.5 s from that cache. The entry names the head, the table and the towers in a new assets field, which a kit built
    before it ignores. Its kind is the new jointDecision (a kit built before it decodes the entry as unknown and
    leaves it out of available(_:); decision would list it in a shipped kit's systemone models, and loading it
    there would download the 15.9 GB decoder for a TypedDecisions that cannot read it). TypedDecisions refuses it by
    its kind, and a decision entry that says jointHead by its format, before downloading anything. /v1/systemone
    gains images (one image: a data URL, base64, or a file path, which SystemOneServer reads only on 127.0.0.1) and
    grid. systemone serve --model clef-flash and the MCP server load it through the new SystemOne.backend(catalog:);
    decide-cli ask and decide-cli serve load KitClefDecider when the entry's kind is jointDecision. A backend that
    reads no images answers an image request with a 422 that names it.
    On an M4 Max through decide-cli parity, downloaded from the catalog pin: 240 of the zoo fixture's runs (200 on its
    186 records, 40 on the 30 held out; its 14 native runs, at the processor's own grid, have no tower here — the zoo's
    gate fed 13 of them the author's image rows, photo_01's 1,200 image rows being more than the decoder's 1,024-row
    buffer), ids and spans equal to the author's on every run, the argmax equal to the author's fp32 read-out on every
    question with a top-2 margin above 0.02 (352/352 and 186/186; near-ties 3/3 and 2/2), max |Δp| 0.0122 and 0.0082, and
    logits, probabilities, the decoder's hidden rows and the responses bit-equal to the zoo's Swift run of the same files
    on all 240. systemone serve --model clef-flash passes conformance/check.py (22 requests and the three routes), and
    the fixture's receipt sent in images, as a data URL and as a path, answers with the zoo Swift run's probabilities to
    the wire's four decimals (14/14 values). ClefFlashTests (the catalog entry, the wire, the request in the author's
    form; with KIT_CLEFFLASH_GATE=1, twelve fixture runs bit for bit against the zoo's Swift run).

  • decider-2b-vision — catalog id decider-2b-vision (the new kind: visionDecision), driven by the new
    KitVisionDecider (Sources/CoreAIKit/DeciderVision/) and CoreAI.decide(image:_:_:grid:options:) (one question,
    a dictionary or an array): typed decisions about an image — Mapika's decider transplanted into Qwen3.5-2B's
    vision-language model (Apache-2.0). Choice, score and noul questions about an image and a state, every question of a
    call read in one pass at its own answer slot, up to 10 options. The host is the model zoo's apps/DeciderVision
    library: the image resized in Pillow's pass order to a fixed grid (g256, 64 image tokens; g448, 196), a vision
    tower per grid (fp16 weights, fp32 math) and a two-function decoder (int8 with three fp16 layers; S = 1 main, S = 16
    prefill) with the tower's rows as a static input, on the low-level runtime. 3,325 MB with the g256 tower; the
    g448 tower (663 MB) downloads when first asked for. Decision.Timing gains imageSeconds and decoderSeconds
    (nil for the text models). On an M4 Max through decide-cli parity, downloaded from the catalog pin: 108/108 slots
    argmax-identical to the author's fp32 read-out, max |Δp| 0.0103, letter logits bit-equal to the zoo's Swift run;
    the zoo gated the same files on an iPhone 18 Pro (device JIT, 108/108; the app needs the increased-memory-limit
    entitlement), the kit has not run on a phone. Not behind /v1/systemone: the wire form has no image field.
    DeciderVisionTests (the fixture's 76 prompt rows against the author's ids, with the tokenizer),
    DeciderVisionSmokeTests (opt-in via KIT_DECIDERVISION_*).

  • Audio8-TTS-Preview-0.6b — catalog id audio8-tts-preview-0.6b (kind: tts), driven by the new
    Audio8TTS (Sources/CoreAIKit/Audio8TTS/): Edge0's DualAR text-to-speech (Fish Audio S2 Pro design), 11
    languages, zero-shot voice cloning from a 0.5–30 s reference (Audio8Voice: the reference's codec codes +
    transcript). One Core AI call per 46 ms frame: the slow AR step, the publisher's top-k / top-p / temperature
    Gumbel-max draw with the RAS rule and the fast AR's ten rows run inside the frame graph; the host supplies the
    uniform draws (Audio8NoiseSource, seeded by default) and streams 32-frame chunks through the causal codec.
    KitSpeaker(catalog: "audio8-tts-preview-0.6b") dispatches it (44.1 kHz). Gated in the zoo on 18 fixtures against
    the publisher's fp32 code (draws 1,671 / 1,695 semantic and 14,711 / 15,093 codebook, Fun-ASR round trip at the
    oracle's level) and on an M4 Max in Release (about real time, RTF 1.05 measured under other load); iPhone not yet
    measured. Audio8SmokeTests (opt-in via KIT_AUDIO8_*).

  • Fun-ASR-Nano-2512 — catalog id fun-asr-nano-2512 (kind: asr), driven by the new
    KitFunASRModel (Sources/CoreAIKit/FunASR/): a kaldi-fbank + LFR front end, funasr's ChatML prompt
    with the audio rows as vocab + slot ids (hotwords, language and itn options) and the decoder's
    63-row audio_embeds static input, like the Qwen3-ASR path. KitTranscriber(catalog:) dispatches
    it. Chinese (dialects, Cantonese), English and Japanese; encoder fp16-weights/fp32-math 450 MB +
    decoder int8 759 MB, both JIT .aimodels for macOS and iOS. Gated in the zoo on 155 clips against
    the fp32 funasr oracle (150 exact, 5 knife-edge, 0 above the margin floor) and on an iPhone 18 Pro
    (device JIT, RTF 0.075, first load 5.1 s then 0.9 s); FunASRSmokeTests (opt-in via KIT_FUNASR_*).

  • 8-speaker diarization: nemotron-3-diarization. NVIDIA's Nemotron-3-Diarization
    (OpenMDW-1.1) tracks up to 8 speakers at 10 ms. It runs behind the same KitDiarizer as
    sortformer-diar-v2 (4 speakers at 80 ms): KitDiarizer(catalog: "nemotron-3-diarization"),
    or MeetingTranscriber(diarizer: "nemotron-3-diarization"). CoreAI.transcribeMeeting and
    MeetingTranscriber's default stay on sortformer-diar-v2.

    The download is 200 MB on either platform: the GPU graph, run in low latency (0.72 s chunks,
    0.32 s look-ahead), and a host/ folder beside it with the projection, the silence row, the mel
    filterbank and the Hann window. The host code is the model zoo's Swift port, copied into
    Sources/CoreAIKit/Audio/NemotronDiarizer. Against transformers' fp32 reference, speaker
    activity agrees on 99.9987 % of a 97.6 s clip and 100 % of a 21.5 s clip. A chunk takes
    15.4–15.9 ms on an M4 Max GPU, measured with another job on the GPU, and 30.1 ms on an iPhone
    17 Pro GPU. NemotronDiarizerTests checks one speaker-cache update and one mel chunk bit for
    bit against the zoo's host, with no weights and no audio.

    SpeakerSegment now carries its frameSec, 0.08 unless given. Each diarizer reports its own
    nSpk and frameSec. diarize(samples:bridgeFrames:) defaults to nil, which means 0.48 s
    in the model's frames. The Sortformer path gives the same results as before: bit-identical
    activity and the same turns on both clips. Examples/Meeting gains --diarizer,
    --asr system and diarize-gate, which checks a diarizer against a reference's per-frame
    probabilities.

  • Zero-shot text classification: gliner2.5-decide. fastino's GLiNER2.5-Decide (Apache-2.0)
    answers any tasks with any labels, named at call time, from one forward: TextClassifier()
    downloads it (ModelID.gliner25Decide), classify(_:tasks:) returns every label's probability
    per task. The host follows gliner2 2.0.0's collator and decision rule; textclassify-cli --gate
    checks both against the zoo's fixtures. Before this, TextClassifier loaded only a local export.
    Examples/TextClassify downloads the catalog model when no --bundle is given. The
    entry's kind is the new textClassification (a kit built before it decodes the entry as
    unknown and leaves it out of available(_:); decision would list it in systemone models,
    whose TypedDecisions cannot load it). Download: 1,756 MB on macOS and on iOS.

  • Examples/TextClassify gains the inbox app. A support inbox of synthetic messages, sorted on
    one tap: every message gets an intent, an urgency and a sentiment from one forward of
    gliner2.5-decide, and the counts and the measured time (the median milliseconds per message,
    messages per second, the total) are on screen, on iPhone and Mac. textclassify-cli --inbox <count> sorts the same inbox headless through the same InboxSorter and prints the same
    numbers. The inbox is generated from a seed (an invented shop, plain item names, first names
    only). -autoplay 1 -log 1 runs it unattended and writes the result as JSON, for a recording or
    a phone read back over devicectl. 1,000 messages took 40.9 s on an iPhone 18 Pro (iOS 27.0,
    GPU, 39.2 ms median per message; the phone holds about 38 ms for the first 20 s of a run and
    slows from there, 1.25–1.5× by 41 s over five runs) and 30.1 s on an M4 Max (30.0 ms),
    2026-09-26.

Fixed

  • VibeVoice, VoxCPM and VoxCPM2 stop at the end of their KV cache. Their decode graphs hold a
    fixed KV cache of 512 positions and write it at each step's position, and the kit did not stop
    before the end: a VibeVoice voice with a long prompt (de-Spk0_man takes 510 positions, kr-Spk1_man
    421) or a VoxCPM text whose tokens and frames came to 512 or more ran steps outside the cache, and
    what came out of them was never checked. StatefulGraphModel now reads the cache's length from
    the graph's state (capacity), and step throws VisionError.kvCacheFull for a pos at or past
    it. VoxCPM and VoxCPM2 end the clip where the cache ends (the text's tokens and the clip's frames
    take one position each, and the decode after the last frame, which nothing read, no longer runs)
    and throw VoxCPMError.textTooLong for a text of 511 tokens or more; a long text ends early, so
    pass one sentence at a time. VibeVoice ends a sentence where a cache ends, so with kr-Spk1_man a
    long sentence ends early, and throws VibeVoiceError.voiceFillsCache for de-Spk0_man, whose
    prompt leaves 2 positions. The en-* voices, the default ones, stay inside the cache. Inputs that
    stay inside the cache give the same output as before.

  • Unlimited-OCR stops at the end of its KV cache. Its decoder holds a fixed KV cache of 2,048
    positions and the image prompt takes 115 of them, and the kit did not stop before the end: a
    maxTokens above 1,933 ran decode steps outside the cache, and what came out of them was never
    checked. KitDocReader now reads the cache's length from the decoder's state and stops a read at
    1,933 tokens, and a decode step at or past the end throws VisionError.kvCacheFull. The default
    maxTokens of 1,024 stays inside the cache; a read that stays inside it returns the same text
    as before.

  • A graph compiled for one iPhone is picked only on that iPhone. The loaders that pick their
    own graph took an AOT .aimodelc over the JIT .aimodel on iOS, most of them by the name
    .h18p.aimodelc, the iPhone 17 Pro's. The iPhone 18 Pro (h19p) refuses that graph
    (incompatibleCompiledAssetArchitecture) and falls back to nothing, even with the JIT graph
    beside it. GraphBundle is now the one rule, and the diarizers, the separator, the forecaster,
    VibeVoice, VoxCPM, VoxCPM2, the Nemotron and Parakeet ASR loaders and the encoder decider use it:
    the AOT graph when it was compiled for this device (AIModel.deviceArchitectureName against the
    <arch> of <name>.<arch>.aimodelc, or of the <function>-<arch> files inside), else the JIT
    .aimodel, else the kit's error naming the architectures the bundle was built for
    (KitBundleError.noGraphForDevice). The Mac follows the same rule; GraphBundle used to prefer
    any .aimodelc there too. VibeVoicePaths.inBundleDir now throws; the aot(root:arch:) builders of
    VoxCPM, VoxCPM2, PocketTTS and DotsTTS default to this device's architecture (was h18p), and
    StableAudioPaths.resolve(root:aot:) names it. From the catalog, VibeVoice, VoxCPM and VoxCPM2
    still load on the iPhone 17 Pro only, as in 0.7.3: their pins have not moved, and ios/ there
    holds only that phone's graphs. On another iPhone they now throw that error.

  • Four catalog models name their JIT graph on iOS. sortformer-diar-v2,
    nemotron-3-diarization, melband-roformer-vocal and timesfm-2.5-200m pointed iOS at the
    iPhone 17 Pro's AOT graph, which only that phone loads. They now name the .aimodel the Mac
    downloads, which an iPhone specializes on its first load (Nemotron-3-Diarization on an iPhone 18
    Pro: 0.78 s the first time, 0.16 s after). ModelID.sortformerDiarV2 names it on both
    platforms. Whisper and VibeVoice keep ios as their path; at their pins it still holds the
    iPhone 17 Pro's graphs (Nemotron 3.5 ASR's pin moves to the JIT graphs, below).

  • The Nemotron-3-Diarization graph compiles for x86_64 again. N3DGraph read and wrote
    Float16 arrays without the guard the other ten files carry
    (#if !((os(macOS) || targetEnvironment(macCatalyst)) && arch(x86_64))), so a Release build of
    any macOS app linking CoreAIKit failed on the Intel slice ('Float16' is unavailable in macOS).
    The diarizer is new in this release, so no released version had this failure. On an Intel Mac a
    float16 N3D bundle now throws the contract error instead; the float32 bundle is unaffected.

  • Four catalog pins move to the JIT layouts. timesfm-2.5-200m, vjepa2-vitl-ssv2,
    nemotron-3.5-asr-streaming-0.6b and gliner2.5-decide are pinned to the Hub revisions of
    2026-09-26 whose ios/ holds the JIT graphs (the compiled ones moved to ios-h18p/,
    GLiNER2.5-Decide's to ios-h19p/), so an iPhone of any generation loads them; sizeMB follows
    (GLiNER2.5-Decide on iOS 1,958 → 1,756 MB). A kit built before this release reads those four
    with its old rule too. vibevoice-realtime-0.5b and voxcpm-0.5b stay on their previous
    revisions: the old loaders open ios/<name>.h18p.aimodelc by name, and the live catalog.json
    reaches every shipped kit, so moving those two pins would stop them on the iPhone 17 Pro. They
    move in a later catalog update (the two Hub repos already carry the JIT layout, b4480f87 and
    38fc0aea); from then an app built with 0.7.3 or earlier no longer loads these two on the iPhone
    17 Pro, while 0.8.0's loaders take the JIT graphs on any iPhone. voxcpm2-2b also stays on its
    previous revision (e03d15d1).

  • ModelID.gliner2PII is pinned. Its ios/ carried the iPhone 17 Pro's compiled files beside
    the JIT graph, which the iPhone 18 Pro refuses; since 2026-09-26 it holds the JIT graph alone
    (627 MB). The preset read main, and an app that had downloaded the old ios/ kept it, since
    the cache is keyed by revision; the pin (74fb5c1) moves the cache path.

  • InformationExtractor opens a graph compiled for this device. It looked for a .aimodel
    alone, and GLiNER2-PII's ios-h18p/ at that pin holds the iPhone 17 Pro's .h18p.aimodelc and
    no .aimodel, so the folder could not be opened. A *.<arch>.aimodelc compiled for this device
    now wins over the .aimodel, as in GraphBundle; a folder with the JIT graph alone loads as
    before.

  • An iPhone takes the graphs its repository compiled for it. A model whose path is ios (the
    catalog's iOS variants, and the presets that leave path nil) always downloaded ios/, so an
    ios-<arch>/ folder, the same bundle compiled for one iPhone generation, was reached only by a
    caller who wrote its path. ModelStore now lists ios-<arch>/ first, <arch> being
    AIModel.deviceArchitectureName (h18p on an iPhone 17 Pro, h19p on an 18 Pro), and takes it
    when the Hub has files there; on a 404 or an empty tree it takes ios/, one listing more. Any
    other error of that listing is thrown, not read as absence. A copy already on disk is used,
    whichever folder it is, and the Hub is not asked, so an ios/ downloaded before stays in use;
    localURL(for:), delete(_:) and the offline fallback look in ios-<arch>/ first.
    remoteSize(of:) and downloadPlan(for:) follow the folder a download takes; the catalog's
    sizeMB stays the ios/ figure, and an ios-<arch>/ can be larger. At the current pins an
    iPhone 17 Pro takes ios-h18p/ for vjepa2-vitl-ssv2, nemotron-3.5-asr-streaming-0.6b and
    ModelID.gliner2PII, and an iPhone 18 Pro takes ios-h19p/ for gliner2.5-decide. Every other
    path, and the Mac, is unchanged. On iOS CoreAIKitCore now imports Core AI, for the
    architecture's name alone.

Known issues

  • On an iPhone 18 Pro with iOS 27.2 (build 24B5099f), Qwen3 0.6B (qwen3-0.6b) answers with
    unreadable text from its first token, and ChatDemo's entry check fails. 0.7.3 gives the same
    text, so this is not new in 0.8.0. The cause is not known. Only this phone and OS build were
    tried. The phone compiles the model's iPhone build (ios/) the first time it loads; not checked
    yet: a build compiled ahead of time for the iPhone 18 Pro, a forced recompile on the phone,
    other iPhones, other iOS builds, and running on the GPU only. On a Mac with macOS 27.0 the
    model answers correctly.