v1.31.9

Release Notes - v1.31.9

Release Date: August 22, 2026

Overview

Version 1.31.9 brings word-level timestamps to Bucky transcription and cancellable native generation to Malina, along with dependency updates to the bucky and malina libraries and supporting documentation.

Detailed Changes

New Features

Feature Bucky Word-Level Timestamps

• Author: William Kennedy
• New WithWordTimestamps transcription option plus a Word type and Transcription.Words field that surface per-word timing from whisper.cpp.
• collectWords and appendWordPiece assemble word pieces from per-token timestamps, splitting and merging on whitespace and dropping empty, non-UTF-8, and EOT tokens.
• The verbose_json transcription response now returns a populated words array (word, start, end) when word granularity is requested, instead of an empty list.

Improvements

Improvement Malina Cancellable Generation

• Author: William Kennedy
• Canceling a request after native generation starts now interrupts the stable-diffusion.cpp call and resets the model context for reuse, instead of waiting for native code to finish.
• A new context support check at load time rejects any loaded context that does not support image generation.
• Cancellation runs in a dedicated goroutine and joins the native, stop, and cancel errors before returning, so neither a canceled context nor a freed handle is reused while native code is active.

Documentation

Documentation Bucky and Malina Manual Updates

• Author: William Kennedy
• Chapter 18 (Bucky) documents the word-level words array in verbose_json and the updated behavior of timestamp_granularities[]=word.
• Chapter 19 (Malina) and README updated to describe cancellable native generation and the new whisper.cpp / bucky 1.1.0 compatibility row.

Dependencies

Dependency Library Updates

• Author: William Kennedy
• bucky v1.0.8 → v1.1.0 (enables word-level timestamps and the NoState model init).
• malina v1.0.3 → v1.0.4 (enables cancellable native generation and context support checks).
• Indirect bumps: GCP OpenTelemetry operations (detectors/gcp v1.35 → v1.36, exporter/metric v0.59 → v0.60), aws smithy-go v1.27.8 → v1.27.9, and googleapis gax-go/v2 v2.23 → v2.24.

Statistics

Category	Count
Total Commits	4
New Features	1
Improvements	1
Bug Fixes	0
Documentation	1

Contributors:

• William Kennedy

Upgrade Notes

Breaking Changes

NONE

Migration

No migration required

Recommended Actions

• Update to bucky v1.1.0 to use WithWordTimestamps and the populated words array in verbose_json.
• Review Malina Generate cancellation behavior: a canceled request now resets the native context for reuse.
• Verify your stable-diffusion contexts support image generation, as unsupported contexts are now rejected at load time.

v1.31.8

Release Notes - v1.31.8

Release Date: August 21, 2026

Overview

Version 1.31.8 brings explicit control over where a multimodal model's media projector (mmproj) runs, support for the new Ornith 1.5 model line, and a faster, lighter test suite. The highlight is a new proj-device option that lets you pin the projector to a specific accelerator (for example, CUDA1 or MTL0), a general upgrade to Yzma v1.24.0, and a large reduction in the test-model download footprint from about 98 GiB to about 52 GiB.

Detailed Changes

New Features

Feature Projector Device Control

• Author: William Kennedy
• New WithProjDevice SDK option and proj-device model-config setting that pins the multimodal projector to a specific backend device (for example, CUDA1 or MTL0); omit for automatic placement.
• proj-device conflicts with proj-on-cpu: true; the language model's device selection via ngpu-layers/main-gpu is unaffected.
• DeviceInfo now reports description, hardware_type, and backend, so kronk devices exposes richer detail and the projector can target the reported name.

Feature Code-Generation Benchmark Harness

• Author: William Kennedy
• New structured code-generation benchmark (sdk/kronk/tests/benchmarks) that runs a two-turn tic-tac-toe coding task and produces token/throughout/TTFT/through speculative-decoding metrics and model-memory estimates.
• Integrated automated grader (grader_test.go) that builds, vets, checks Go structure, and runs win/draw/undo scenarios against each response.
• New make targets (benchmark-codegen-*) with BENCH_CODEGEN_TIME and BENCH_CODEGEN_TIMEOUT knobs, replacing the previous bench suite.

Improvements

Improvement Ornith 1.5 Model Support

• Author: William Kennedy
• Agent model config now uses kronk/ornith-ai/Ornith-1.5-35B-Q8_0/AGENT as the main model, retiring the Ornith 1.0 35B entry.
• Updated model capabilities, context/output limits, and catalog/model-config entries to match the Ornith 1.5 line.
• Adjusted small-model and additional-model entries and their limits in .agents.

Improvement Smaller Test-Model Footprint

• Author: William Kennedy
• Test-model set reduced from about 98 GiB to about 52 GiB, using smaller quantizations that still exercise each runtime path.
• Cleaned the default catalog (sdk/tools/defaults/yaml/catalog.yaml) by removing stale/unused model entries.
• Added MODEL_USAGE.md to record the intent behind each model-backed test, benchmark, and example selection.

Improvement CI Model-Backed Coverage

• Author: William Kennedy
• GitHub Actions now run focused, model-backed tests per capability in linux.yml (ThinkChat, tool-call content, draft/speculative, vision, rerank) instead of relying solely on local runs.
• Added small test models (Qwen3-0.6B-Q8_0, Qwopus3.5-4B-Coder.Q4_K_M) and trimmed models to fit the GitHub Actions 10 GiB cache quota.
• Refined kronk launch model filtering for cleaner regeneration of the OpenCode config.

Improvement GPU Device Classification

• Author: William Kennedy
• ClassifyDeviceType now classifies by backend device metadata (hardware type + registration name), giving more accurate gpu_rocm/gpu_cuda/gpu_metal/gpu_vulkan mapping.
• Refactored ROCm/HIP alias handling so it no longer depends solely on by-name prefixes.

Library Updates

• Author: William Kennedy
• Upgraded Yzma v1.23.0 → v1.24.0, bringing the underlying llama.cpp default library from b10520 → b10549.
• Refreshed transitive dependencies (grpc, aws-sdk-go-v2, cloud.google.com/storage, genproto, testify, logrus, mewkiz/flac, lestrrat-go/dsig).

Bug Fixes

None.

Documentation

Documentation Projector + Benchmark Docs

• Author: William Kennedy
• Chapter 03 documents the new proj-device setting, its conflict with proj-on-cpu, and the updated config-reference table.
• New code-generation benchmark README documents the protocol, metrics, make targets, and artifacts.
• Updated several manual chapters and BUI docs (DocsManual, DocsSDKModel) to reflect the cleaned model list and projector options.

Dependencies

Dependency Yzma and Transitive Dependencies

• Author: William Kennedy
• github.com/hybridgroup/yzma v1.23.0 → v1.24.0 (primary library upgrade).
• kronk libs default llama.cpp build bumped b10520 → b10549.
• Bumped grpc, aws-sdk-go-v2 suite, cloud.google.com/auth and storage, genproto packages, testify, sirupsen/logrus, mewkiz/flac, and lestrrat-go/dsig.

Statistics

Category	Count
Total Commits	4
New Features	2
Improvements	4
Bug Fixes	0
Documentation	1

Contributors:

• William Kennedy

Upgrade Notes

Breaking Changes

None.

Migration

No migration required. The new proj-device setting is optional; omit it to keep the previous automatic projector behavior.

Recommended Actions

• Review the cleaned default catalog and test-model list if any of those entries were relied upon.
• Optionally add proj-device to pin the multimodal projector on multi-GPU systems.
• Re-run any custom benchmark scripts against the new benchmark-codegen-* make targets.

v1.31.7

Release Notes - v1.31.7

Release Date: August 19, 2026

Overview

Version 1.31.7 brings model configuration updates, improvements to media message normalization that preserves message fields, and removes the enum restriction on reasoning_effort to support newer templates.

Detailed Changes

Improvements

Improvement Media Message Field Preservation

• Author: William Kennedy
• toMediaMessage now preserves reasoning_content, tool_calls, and tool_call_id fields instead of dropping them during media normalization
• Messages are cloned before normalization so only content fields are rewritten
• A length mismatch check between source and normalized messages prevents silent data loss
• Non-user roles (system, assistant, tool) are now correctly preserved through the full message lifecycle

Improvement Removed reasoning_effort Enum Restriction

• Author: William Kennedy
• Removed the hardcoded enum validation for reasoning_effort in parseReasoningString
• The SDK now accepts any string value so newer chat templates can add reasoning levels without requiring a server change
• Invalid values are passed through to the template which may reject them

Improvement Model Configuration Updates

• Author: William Kennedy
• Added unsloth/Ornith-1.0-35B-UD-Q8_K_XL/AGENT model config with nseq-max: 2, context-window: 131072, and preserve_thinking: true
• Removed hardcoded temperature, top_k, and top_p overrides for all AGENT models to use their GGUF sampling defaults
• Added chat-template-kwargs: preserve_thinking: true to all AGENT models

Bug Fixes

Bug Fix Removed Invalid reasoning_effort Test Case

• Author: William Kennedy
• Removed test case that expected "invalid" to fail validation on reasoning_effort
• The new behavior allows any string so templates can validate their own values

Documentation

Documentation Updated Request Parameters Manual

• Author: William Kennedy
• Updated Chapter 10 to document the flexible reasoning_effort behavior
• Updated API Endpoints manual with catalog and operations endpoint details

Statistics

Category	Count
Total Commits	1
New Features	0
Improvements	3
Bug Fixes	1
Documentation	1

Contributors:

• William Kennedy

Upgrade Notes

Breaking Changes

None

Migration

No migration required

Recommended Actions

• Review model configuration files if custom sampling overrides were set — defaults now come from the model's GGUF metadata
• Verify that any custom reasoning_effort values are compatible with your chat templates

v1.31.6

Release Notes - v1.31.6

Release Date: August 19, 2026

Overview

Version 1.31.6 brings a concurrency redesign to Malina (image generation), a new kronk malina CLI subcommand tree for managing stable-diffusion.cpp libraries and model bundles, improved Bucky admission timeout handling, and a Go toolchain bump to 1.27.0.

Detailed Changes

New Features

Malina CLI Subcommands

• Author: William Kennedy
• New kronk malina libs command for installing, listing, and removing stable-diffusion.cpp library bundles
• New kronk malina model command tree with catalog, list, pull, and remove subcommands for managing curated image-model bundles
• Library and model management commands require --local flag since Malina routes are not yet served by the Kronk model server

Improvements

Malina Concurrency Redesign

• Author: William Kennedy
• Malina now uses a pool of backend contexts instead of a single backend, enabling concurrent image generation across multiple model contexts
• Each backend still performs one generation at a time (stable-diffusion contexts are not safe for concurrent use)
• Admission channel capacity scaled to Concurrency + QueueDepth for proper backpressure across the pool
• Workers use sync.WaitGroup for clean shutdown coordination

Bucky Admission Timeout

• Author: William Kennedy
• Admission wait is now independently bounded by AdmissionTimeout, separate from the caller's original context
• Introduces ErrAdmissionTimeout error variable so callers can distinguish admission timeouts from context cancellation
• Once admitted, model processing continues under the caller's original context

Go Toolchain Bump

• Author: William Kennedy
• Updated minimum Go version from 1.26.6 to 1.27.0 across the project

Bug Fixes

Bucky Concurrency Surface

• Author: William Kennedy
• Fixed Bucky API concurrency surface to correctly use the new admission channel pattern

Documentation

Bucky and Malina Docs

• Author: William Kennedy
• Updated Bucky documentation with sampling parameters
• Updated Malina documentation reflecting concurrency redesign and new CLI commands

Dependencies

Project Image and Infrastructure

• Author: William Kennedy
• Updated project banner image
• Updated installation, model configuration, and developer guide documentation
• Updated Nix flake.lock and flake.nix for dependency consistency

Statistics

Category	Count
Total Commits	3
New Features	1
Improvements	3
Bug Fixes	1
Documentation	2

Contributors:

• William Kennedy

Upgrade Notes

Breaking Changes

None

Migration

No migration required

Recommended Actions

• Update Go toolchain to 1.27.0
• If using Malina with high concurrency, review the new pool-based architecture — the single-backend model is replaced by a pool of Concurrency backends
• If using Bucky and relying on context cancellation for admission backpressure, check for ErrAdmissionTimeout to distinguish admission timeouts from other cancellations

v1.31.5

Release Notes - v1.31.5

Release Date: August 19, 2026

Overview

Version 1.31.5 introduces comprehensive whisper.cpp sampling parameters for both transcription and streaming modes in Bucky, allowing fine-grained control over decoding behavior including temperature, beam search, and greedy strategies. Documentation and BUI components have been updated to reflect these new capabilities.

Detailed Changes

New Features

Bucky Sampling Parameters

• Author: William Kennedy
• Added Temperature, TemperatureInc, EntropyThreshold, GreedyBestOf, BeamSize, BeamSearchPatience, and LengthPenalty fields to TranscribeConfig in sdk/bucky/model/transcribe.go
• Added the same sampling parameters to StreamConfig in sdk/bucky/model/stream.go for streaming transcription sessions
• Introduced constructor options: WithTemperature, WithTemperatureInc, WithEntropyThreshold, WithGreedyBestOf, WithBeamSearchPatience, and WithLengthPenalty for both transcribe and stream configs
• Added parseWhisperOptions function in cmd/server/app/domain/audioapp/audioapp.go to parse whisper options from HTTP form values, exposing all new parameters through the API endpoint
• Added comprehensive test coverage for sampling parameters in sdk/bucky/model/sampling_params_test.go

Documentation

Documentation Bucky Sampling Parameters

• Author: William Kennedy
• Updated .manual/chapter-03-model-configuration.md with documentation for the new sampling parameters
• Updated .manual/chapter-04-batch-processing.md with additional context
• Updated .manual/chapter-18-bucky.md with details on the new Bucky sampling parameters
• Updated BUI components DocsManual.tsx and DocsSDKBuckyModel.tsx to reflect the new documentation
• Updated static assets for the Browser UI (BUI)

Statistics

Category	Count
Total Commits	1
New Features	1
Improvements	0
Bug Fixes	0
Documentation	1

Contributors:

• William Kennedy

Upgrade Notes

Breaking Changes

None

Migration

No migration required

Recommended Actions

• Review the new Bucky sampling parameters in the SDK and API documentation
• Update transcription and streaming calls to leverage the new decoding controls for improved accuracy

v1.31.4

Release Notes - v1.31.4

Release Date: August 18, 2026

Overview

Version 1.31.4 introduces canonical model identifiers across all APIs, a new immutable System cache pool for the Incremental Message Cache, improved pool auto-tuning with weighted multi-GPU budget allocation, and model server configuration support. It also includes batch engine optimizations, documentation updates, and dependency upgrades.

Detailed Changes

New Features

Canonical Model Identifiers

• Author: William Kennedy
• Model identifiers now use the canonical provider/modelID form on all public model-facing APIs
• Bare model IDs are rejected instead of being searched across a provider list
• Model pull and catalog resolution no longer walk a configured provider list for bare IDs
• Discovery responses (GET /v1/models) and inference responses now report canonical IDs as the source of truth
• Named configurations append the profile after the canonical base ID (e.g., Qwen/Qwen3-8B-Q8_0/AGENT)
• New ParseModelID and ModelID types in sdk/tools/models for canonical identifier parsing

IMC System Cache Pool

• Author: William Kennedy
• New immutable System preload pool for Incremental Message Cache (IMC)
• Dedicated IMCSystemCaches() API returns snapshots of System cache pool entries
• System caches persist across sessions for shared prompt prefixes and template content
• Exposes per-entry details: token count, allocated size, snapshot bytes, restore count, and last-used time
• Significantly reduces redundant prefill work for common system prompts

Model Server Settings in Config

• Author: William Kennedy
• Model server now supports per-model configuration via model_config.yaml
• New config.go service in the API server for structured config management
• Configuration supports context window, split mode, and other model-level overrides
• Legacy bare model ID keys in config are rejected in favor of canonical identifiers

Improvements

Pool Auto-Tune Weighted Budget

• Author: William Kennedy
• Auto-tune now uses weighted multi-GPU budget allocation based on tensor split or device capacity
• ResponseModelID is now propagated from pool load requests through to model config
• GPU budget calculation considers tensor split weights when available, falling back to device total bytes
• Improved device selection logic respects explicit device configuration over usage-based discovery

Batch Engine Optimizations

• Author: William Kennedy
• IMC optimization changes across the batch generation engine
• Improved prefill media handling and slot management
• Enhanced speculative decoding integration with batch engine
• Better diagnostics reporting for batch generation state
• Optimized token handling and cache management in prefill paths

BUI IMC Sessions UI

• Author: William Kennedy
• Refreshed IMC Sessions screen in the Browser UI with improved layout and information display
• Updated API types and service integration for IMC session details
• Improved CSS styling for sessions component

Bug Fixes

Model ID Resolution

• Author: William Kennedy
• Fixed model ID resolution in model pull and resolve commands to use canonical identifiers
• Corrected model ID handling in launch command and server startup
• Fixed model ID propagation through pool and tool app domain layers

AuthTune Configuration

• Author: William Kennedy
• Fixed AuthTune configuration handling in server startup

Qwen Tool Parsing

• Author: William Kennedy
• Fixed tool parsing in Qwen parser for improved compatibility

Documentation

Comprehensive Updates

• Author: William Kennedy
• Updated installation guide with latest model configuration details
• Expanded model configuration documentation including server settings
• Updated batch processing documentation for IMC optimizations
• Enhanced speculative decoding and MTP documentation
• Updated YaRN extended context documentation
• Expanded model server documentation with config file details
• Updated API endpoints documentation
• Improved troubleshooting guide
• Updated Malina (image generation) documentation
• Refreshed developer guide with latest build and architecture information
• Updated observability documentation for Prometheus metrics
• Updated BUI documentation
• Removed outdated IMC diagram SVGs and refreshed session selection diagram

Dependencies

Dependency Updates

• Author: William Kennedy
• github.com/ardanlabs/malina v1.0.2 → v1.0.3
• github.com/open-policy-agent/opa v1.19.0 → v1.19.1
• cel.dev/expr v0.25.2 → v0.25.3
• Full AWS SDK for Go v2 dependency chain updated (v1.43.5 → v1.43.6 and related sub-modules)
• github.com/aws/smithy-go v1.27.7 → v1.27.8

Statistics

Category	Count
Total Commits	4
New Features	3
Improvements	3
Bug Fixes	1
Documentation	1

Contributors:

• William Kennedy

Upgrade Notes

Breaking Changes

Model Identifier Changes

Kronk model identifiers now use the canonical provider/modelID form on all public model-facing APIs. Bare model IDs are rejected instead of being searched across a provider list.

Update the model field sent to Chat Completions, Responses, Messages, Embeddings, Reranking, Tokenize, and other inference endpoints. Also update model IDs supplied to model-management APIs and CLI commands.

Keys in ~/.kronk/models/model_config.yaml must also use canonical identifiers. A legacy bare key prevents the server pool from initializing.

Named configurations now append the profile after the canonical base ID. For example, replace Qwen3-8B-Q8_0/AGENT with Qwen/Qwen3-8B-Q8_0/AGENT.

See BREAKING_CHANGES.md for full migration details.

Migration

Update all model identifiers to canonical provider/modelID form. Update model config YAML keys. See BREAKING_CHANGES.md for complete migration instructions.

Recommended Actions

• Update all model identifiers in API requests and configuration files to canonical provider/modelID form
• Verify model config YAML uses canonical keys (run server to confirm pool initializes)
• Review BREAKING_CHANGES.md for detailed migration guidance
• Test IMC behavior with the new System cache pool to verify performance improvements

v1.31.3

Release Notes - v1.31.3

Release Date: August 16, 2026

Overview

Version 1.31.3 delivers a major architectural reorganization of the speculative decoding engine, separating it into well-defined packages for classic and MTP (Multi-Token Prediction) strategies. This refactor significantly improves code maintainability and readability while also fixing validation bugs in chat processing.

Detailed Changes

New Features

Speculative Decoding Package Refactor

• Author: William Kennedy
• Separated speculative decoding logic from the batch engine into dedicated packages under sdk/kronk/model/internal/speculation/
• Created classic package implementing classic speculative decoding with generate, verify, and state management components
• Created mtp package implementing MTP-specific draft generation, synchronization, verification, and finalization
• Introduced speculation.Controller interface with Disabled fallback implementation for non-speculative workloads
• Added speculation_runtime.go as the central runtime controller that dispatches to classic or MTP engines based on configuration
• Added speculation_config.go to centralize speculation configuration parsing and validation
• Added mtp_runtime_hooks.go with MTP-specific runtime hooks for the yzma integration
• All new packages include comprehensive test coverage (disabled_test.go, classic_test.go, mtp/*_test.go, speculation_test.go)

Improvements

Validation Bug Fixes

• Author: William Kennedy
• Fixed validation bugs in speculative decoding configuration and chat processing
• Added 57 new lines of tests in cmd/server/api/services/kronk/tests/chat_test.go for chat validation scenarios
• Added 75 new lines of tests in sdk/kronk/model/chat_test.go for chat processing validation
• Streamlined batch engine code by removing speculative decoding logic (270 lines removed from batchgen_engine.go, 748 lines from batchgen_speculative.go)
• Cleaned up batchgen_slot.go and batchgen_slot_start.go to remove speculative decoding concerns
• Removed speculative_sparse.go (109 lines) as the functionality has been reorganized into the new speculation packages

Documentation

Speculative Decoding Documentation Updates

• Author: William Kennedy
• Updated .manual/chapter-03-model-configuration.md with 12 new lines covering speculation configuration
• Updated .manual/chapter-06-speculative-decoding-mtp.md with 6 new lines reflecting the new architecture
• Updated BUI components (DocsManual.tsx, DocsSDKModel.tsx) with 42 new lines of documentation content

Dependencies

Configuration Tool Updates

• Updated sdk/tools/models/kronkconfig.go and kronkresolve.go to support new speculation configuration fields
• Added tests for kronkresolve configuration parsing
• Updated model config YAML files in sdk/tools/defaults/yaml/ and zarf/kms/

Statistics

Category	Count
Total Commits	2
New Features	1
Improvements	1
Bug Fixes	1
Documentation	1

Contributors:

• William Kennedy

Upgrade Notes

Breaking Changes

None

Migration

No migration required. The speculative decoding functionality remains the same from a user perspective — the changes are internal reorganization of the codebase.

Recommended Actions

• No action required for existing users
• If you have custom speculation configuration, verify it still works with the updated configuration parsing
• Review the updated documentation in Chapter 3 and Chapter 6 for the latest speculation configuration guidance

v1.31.2

Release Notes - v1.31.2

Release Date: August 16, 2026

Overview

Version 1.31.2 brings a major rework of Kronk's batch scheduling engine with new diagnostics, a single prefill-batch-size configuration setting replacing nbatch/nubatch, a Slots UI for real-time scheduler visibility, a new media load testing tool, and a BREAKING change to model batch configuration.

Detailed Changes

New Features

Batch Engine Diagnostics API

• Author: William Kennedy
• New BatchEngineSnapshot type exposes real-time scheduler state per loaded model
• Each snapshot includes per-slot phase, generation mode, token counts, and IMC preparation progress
• Generation contributions track each slot's row count and mode (ordinary, speculative, MTP, M-RoPE direct)
• Prefill and IMC selector cursors show which slots are eligible and currently selected
• Available via Model.BatchEngineSnapshot() in the SDK and GET /v1/kronk/models/{model}/slots on the server

Slots Browser UI

• Author: William Kennedy
• New Slots tab in the BUI displays live scheduler state for all loaded generation models
• Shows per-slot phase (idle, starting, prefill, media-prefill, IMC preparation, generation), token counts, request age, and generation mode
• Model-level summary includes iteration count, batch sizing, MTP/speculative mode, and selector cursors
• Color-coded phase badges and generation mode labels for quick visual scanning
• Auto-refreshes every 2 seconds to keep scheduler state current

Media Load Testing Tool

• Author: William Kennedy
• New .tools/media-load/media-load.py for benchmarking Kronk with multimodal media inputs
• Supports parallel clients, configurable image/audio payloads, and streaming responses
• Generates throughput and latency metrics for vision and audio model workloads

SDK Batch Engine Snapshot

• Author: William Kennedy
• kronk.(*Kronk).BatchEngineSnapshot() exposes the latest scheduler state from the generation batch engine
• Returns (BatchEngineSnapshot, bool) — the bool is false for embedding/rerank models
• Complements IMCSessions() for complete runtime observability

Improvements

Single Prefill Batch Size Configuration

• Author: William Kennedy
• Replaced separate nbatch and nubatch settings with one prefill-batch-size (default: 2048)
• prefill-batch-size limits how many prompt tokens the current prefill owner contributes per decode iteration
• Kronk derives internal NBatch and NUBatch capacities from prefill-batch-size, slot count, and generation mode
• Non-MTP reserves one generation row per slot; MTP reserves 1 + ndraft rows per slot

Round-Robin Prefill Replaced by Prefill Owner

• Author: William Kennedy
• Prefill now uses a persistent owner model instead of round-robin across slots
• One active slot owns prefill until its prompt completes, then the cursor advances to the next eligible slot
• Generation and speculative rows from ready output slots are staged first, then the remaining tray capacity goes to the prefill owner
• This gets long-prompt prefill done quickly without blocking streaming output from other slots

IMC Preparation Scheduling

• Author: William Kennedy
• Text IMC preparation is deferred to its own scheduling phase after slot admission
• Newly admitted requests can claim free slots immediately without waiting for IMC decoding
• If no rows were staged during admission, IMC preparation runs before prefill

Configuration Cleanup

• Author: William Kennedy
• Minor cleanup to configuration handling and edge cases
• Removed unused nbatch/nubatch references from config files and examples

Documentation

Manual Updates

• Author: William Kennedy
• Updated Chapter 4 (Batch Processing) with new prefill owner model, batch sizing diagrams, and selector cursor explanations
• Updated Chapter 3 (Model Configuration) with prefill-batch-size documentation
• Updated Chapter 5 (Message Caching) with new IMC session selection diagram
• Updated Chapter 6 (Speculative Decoding & MTP) with MTP batch sizing notes
• Updated Chapter 11 (Multi-Modal Models) and Chapter 13 (Browser UI)
• Added new SVG diagrams for batch sizing, prefill batching, IMC cache growth, and IMC session selection
• Added BREAKING_CHANGES.md documenting the v1.31.2 batch configuration migration

SDK Documentation

• Author: William Kennedy
• Updated DocsSDKModel.tsx with new batch engine diagnostics API examples
• Updated DocsSDKPool.tsx with pool-level snapshot references
• Updated DocsSDKKronk.tsx with BatchEngineSnapshot() usage

Dependencies

Build Toolchain

• Author: William Kennedy
• Updated .make/tools.mk with toolchain improvements

Statistics

Category	Count
Total Commits	2
New Features	4
Improvements	4
Bug Fixes	0
Documentation	3

Contributors:

• William Kennedy

Upgrade Notes

Breaking Changes

Model Batch Configuration Changes

The separate user-configurable nbatch and nubatch settings were replaced by one prompt-processing setting:

# Before
nbatch: 8192
nubatch: 2048

# After
prefill-batch-size: 2048

prefill-batch-size defaults to 2048 and limits how many prompt tokens the current prefill owner contributes to one decode iteration. Kronk now derives llama.cpp's internal logical NBatch and physical NUBatch capacities from that value, the slot count, and the generation mode. Non-MTP reserves one generation row per slot. MTP reserves 1 + ndraft rows per slot and uses one physical batch for the complete prefill-plus-generation tray.

Direct Go SDK consumers must replace these removed APIs:

Removed	Replacement
model.Config.PtrNBatch	model.Config.PtrPrefillBatchSize
model.Config.PtrNUBatch	model.Config.PtrPrefillBatchSize
model.Config.NBatch()	model.Config.PrefillBatchSize() for configuration; EffectiveNBatch() for diagnostics
model.Config.NUBatch()	model.Config.PrefillBatchSize() for configuration; EffectiveNUBatch() for diagnostics
model.WithNBatch(...)	model.WithPrefillBatchSize(...)
model.WithNUBatch(...)	model.WithPrefillBatchSize(...)

The Playground request field changed from nbatch and nubatch to prefill_batch_size. The BUI playground and configuration sweeps now expose only Prefill Batch Size. Effective NBatch / NUBatch values remain visible as read-only runtime diagnostics on the Slots screen.

Migration

When migrating an old configuration with different values, use the old nubatch value as the initial prefill-batch-size, then benchmark the workload. A larger value can finish long-prompt prefill in fewer decode calls, but each call takes longer before already-generating slots can run again and requires a larger compute buffer.

Recommended Actions

• Replace nbatch and nubatch YAML settings with prefill-batch-size in all configuration files
• Update Go SDK code: replace PtrNBatch/PtrNUBatch with PtrPrefillBatchSize, replace WithNBatch/WithNUBatch with WithPrefillBatchSize
• Replace model.Config.NBatch() calls with PrefillBatchSize() for config or EffectiveNBatch() for diagnostics
• Replace model.Config.NUBatch() calls with PrefillBatchSize() for config or EffectiveNUBatch() for diagnostics
• Update playground request payloads: replace nbatch/nubatch fields with prefill_batch_size
• Explore the new Slots tab in the BUI for real-time scheduler visibility
• Review BREAKING_CHANGES.md for complete migration details

v1.31.1

Release Notes - v1.31.1

Release Date: August 15, 2026

Overview

Version 1.31.1 brings MTP performance improvements, new support for the Qwen3.8-27B model, and documentation updates for Homebrew installation.

Detailed Changes

New Features

Qwen3.8-27B Model Support

• Author: William Kennedy
• Added catalog entry for unsloth/Qwen3.8-27B-UD-Q4_K_XL (Hybrid model with vision, reasoning, and tooling capabilities)
• Added model configuration for unsloth/Qwen3.8-27B-UD-Q4_K_XL/AGENT with reasoning_effort: "low" and 16384 max tokens
• Updated libs.go for compatibility

Improvements

MTP Performance Enhancements

• Author: William Kennedy
• Removed adaptive chooseNDraft throttling for MTP — now uses the configured draft count to match llama.cpp behavior
• Added detailed timing observability for MTP decode operations: target decode elapsed, MTP sync elapsed, decode wait elapsed, and chunk counts
• Updated defMTPNDraft from 2 to 3 to match llama.cpp's MTP default
• Clarified comments distinguishing MTP drafting from separate-GGUF speculative decoding

Reasoning Effort Handling

• Author: William Kennedy
• Removed the global default DefReasoningEffort constant so the chat template determines the default reasoning level
• Updated Mistral parser to only coerce explicit reasoning_effort values — an empty value remains unset, allowing the template's native default to apply
• Added test coverage for Mistral parser reasoning effort handling

Model Config Updates

• Author: William Kennedy
• Increased max_tokens from 8192 to 16384 for AGENT models: gemma-4-26B-A4B, mtp-Qwen3.6-27B, mtp-Qwen3.6-35B-A3B, gemma-4-E4B, and Qwopus3.5-4B-Coder

Documentation

Homebrew Installation Fix

• Author: William Kennedy
• Updated Homebrew installation instructions in .manual/chapter-02-installation.md and README.md

Statistics

Category	Count
Total Commits	3
New Features	1
Improvements	3
Bug Fixes	0
Documentation	1

Contributors:

• William Kennedy

Upgrade Notes

Breaking Changes

None

Migration

No migration required

Recommended Actions

• Review the updated MTP behavior: draft token count no longer adapts based on acceptance rate for auto-detected MTP heads — it uses the configured value (default 3) to match llama.cpp
• If you relied on the previous DefReasoningEffort default of "medium", verify your templates handle the reasoning effort default appropriately

v1.31.0

Release Notes - v1.31.0

Release Date: August 14, 2026

Overview

Version 1.31.0 brings improved error resilience in batch generation, new load mode options, VRAM optimization with per-sequence output limits, GGUF filetype/quantization visibility, and an update to llama.cpp b10423.

Detailed Changes

Improvements

LoadModeAuto Default and LoadModeMMapMLock

• Author: William Kennedy
• Changed default LoadMode from mmap to auto, which uses mmap when every selected device supports it and falls back to ordinary loading otherwise
• Added LoadModeMMapMLock option that combines mmap with resident page locking
• Updated DerefLoadMode to default to LoadModeAuto instead of LoadModeMMap
• Updated config documentation and YAML defaults to reflect the new options

NOutputsMaxPerSeq VRAM Optimization

• Author: William Kennedy
• Added NOutputsMaxPerSeq context parameter for per-sequence logit row reservation
• Prevents a single sequence from claiming the total logit rows, freeing significant VRAM at large ubatch sizes with MTP
• Applied in both generation and IMC prefill paths
• Context pool fallback now respects the per-sequence limit

GGUF FileType and Quantization Visibility

• Author: William Kennedy
• Added FileType and Quantization fields to ModelInfo struct in both SDK and tools
• Model info display now shows file type and friendly quantization name (e.g., "Q4_K - Medium")
• Updated ModelInfoFromMetadata to parse general.file_type from GGUF headers
• Added TestGGMLTypeSizes test covering all supported GGML types including MXFP4, NVFP4, Q1_0, and Q2_0

Batch Generation Error Resilience

• Author: William Kennedy
• Added error handling to all batch.Add() calls across batch generation engine
• MTP draft generation now returns errors properly through the drafter interface
• Speculative decoding batch operations handle errors with proper slot cleanup
• Prefill text and media operations rollback batch state on errors and finish slots cleanly
• Batch sequence engine for embed/rerank models handles add errors

Yzma / llama.cpp Update

• Author: William Kennedy
• Updated default llama.cpp version from b10417 to b10423
• Updated GGML type block sizes for Q2_K, Q8_K, IQ2_XS, IQ3_XXS, and IQ4_XS to match upstream
• Added GGML type definitions for MXFP4 (39), NVFP4 (40), Q1_0 (41), and Q2_0 (42)

Documentation

Model Configuration Docs

• Author: William Kennedy
• Updated Chapter 3 (Model Configuration) with new load mode options
• Updated Chapter 17 (Troubleshooting) with relevant changes
• Updated BUI documentation components to reflect new config options

Dependencies

Dependency llama.cpp Update

• Updated default llama.cpp version from b10417 to b10423
• Uses new download.Install API instead of download.GetWithContext

Statistics

Category	Count
Total Commits	1
New Features	0
Improvements	5
Bug Fixes	1
Documentation	1

Contributors:

• William Kennedy

Upgrade Notes

Breaking Changes

None

Migration

No migration required. The default LoadMode changed from mmap to auto, but this maintains the same behavior when all devices support mmap. Explicit load-mode: mmap in config files will continue to work as before.

Recommended Actions

• Review model configuration files if you have explicit load-mode settings — the default is now auto instead of mmap
• Consider using load-mode: mmap+mlock if you want the combined mmap + mlock behavior that was previously not available as a single option
• Update any code relying on LoadMode zero value being MMap — it is now Auto