v1.31.4

Release Notes - v1.31.4

Release Date: August 18, 2026

Overview

Version 1.31.4 introduces canonical model identifiers across all APIs, a new immutable System cache pool for the Incremental Message Cache, improved pool auto-tuning with weighted multi-GPU budget allocation, and model server configuration support. It also includes batch engine optimizations, documentation updates, and dependency upgrades.

Detailed Changes

New Features

Canonical Model Identifiers

• Author: William Kennedy
• Model identifiers now use the canonical provider/modelID form on all public model-facing APIs
• Bare model IDs are rejected instead of being searched across a provider list
• Model pull and catalog resolution no longer walk a configured provider list for bare IDs
• Discovery responses (GET /v1/models) and inference responses now report canonical IDs as the source of truth
• Named configurations append the profile after the canonical base ID (e.g., Qwen/Qwen3-8B-Q8_0/AGENT)
• New ParseModelID and ModelID types in sdk/tools/models for canonical identifier parsing

IMC System Cache Pool

• Author: William Kennedy
• New immutable System preload pool for Incremental Message Cache (IMC)
• Dedicated IMCSystemCaches() API returns snapshots of System cache pool entries
• System caches persist across sessions for shared prompt prefixes and template content
• Exposes per-entry details: token count, allocated size, snapshot bytes, restore count, and last-used time
• Significantly reduces redundant prefill work for common system prompts

Model Server Settings in Config

• Author: William Kennedy
• Model server now supports per-model configuration via model_config.yaml
• New config.go service in the API server for structured config management
• Configuration supports context window, split mode, and other model-level overrides
• Legacy bare model ID keys in config are rejected in favor of canonical identifiers

Improvements

Pool Auto-Tune Weighted Budget

• Author: William Kennedy
• Auto-tune now uses weighted multi-GPU budget allocation based on tensor split or device capacity
• ResponseModelID is now propagated from pool load requests through to model config
• GPU budget calculation considers tensor split weights when available, falling back to device total bytes
• Improved device selection logic respects explicit device configuration over usage-based discovery

Batch Engine Optimizations

• Author: William Kennedy
• IMC optimization changes across the batch generation engine
• Improved prefill media handling and slot management
• Enhanced speculative decoding integration with batch engine
• Better diagnostics reporting for batch generation state
• Optimized token handling and cache management in prefill paths

BUI IMC Sessions UI

• Author: William Kennedy
• Refreshed IMC Sessions screen in the Browser UI with improved layout and information display
• Updated API types and service integration for IMC session details
• Improved CSS styling for sessions component

Bug Fixes

Model ID Resolution

• Author: William Kennedy
• Fixed model ID resolution in model pull and resolve commands to use canonical identifiers
• Corrected model ID handling in launch command and server startup
• Fixed model ID propagation through pool and tool app domain layers

AuthTune Configuration

• Author: William Kennedy
• Fixed AuthTune configuration handling in server startup

Qwen Tool Parsing

• Author: William Kennedy
• Fixed tool parsing in Qwen parser for improved compatibility

Documentation

Comprehensive Updates

• Author: William Kennedy
• Updated installation guide with latest model configuration details
• Expanded model configuration documentation including server settings
• Updated batch processing documentation for IMC optimizations
• Enhanced speculative decoding and MTP documentation
• Updated YaRN extended context documentation
• Expanded model server documentation with config file details
• Updated API endpoints documentation
• Improved troubleshooting guide
• Updated Malina (image generation) documentation
• Refreshed developer guide with latest build and architecture information
• Updated observability documentation for Prometheus metrics
• Updated BUI documentation
• Removed outdated IMC diagram SVGs and refreshed session selection diagram

Dependencies

Dependency Updates

• Author: William Kennedy
• github.com/ardanlabs/malina v1.0.2 → v1.0.3
• github.com/open-policy-agent/opa v1.19.0 → v1.19.1
• cel.dev/expr v0.25.2 → v0.25.3
• Full AWS SDK for Go v2 dependency chain updated (v1.43.5 → v1.43.6 and related sub-modules)
• github.com/aws/smithy-go v1.27.7 → v1.27.8

Statistics

Category	Count
Total Commits	4
New Features	3
Improvements	3
Bug Fixes	1
Documentation	1

Contributors:

• William Kennedy

Upgrade Notes

Breaking Changes

Model Identifier Changes

Kronk model identifiers now use the canonical provider/modelID form on all public model-facing APIs. Bare model IDs are rejected instead of being searched across a provider list.

Update the model field sent to Chat Completions, Responses, Messages, Embeddings, Reranking, Tokenize, and other inference endpoints. Also update model IDs supplied to model-management APIs and CLI commands.

Keys in ~/.kronk/models/model_config.yaml must also use canonical identifiers. A legacy bare key prevents the server pool from initializing.

Named configurations now append the profile after the canonical base ID. For example, replace Qwen3-8B-Q8_0/AGENT with Qwen/Qwen3-8B-Q8_0/AGENT.

See BREAKING_CHANGES.md for full migration details.

Migration

Update all model identifiers to canonical provider/modelID form. Update model config YAML keys. See BREAKING_CHANGES.md for complete migration instructions.

Recommended Actions

• Update all model identifiers in API requests and configuration files to canonical provider/modelID form
• Verify model config YAML uses canonical keys (run server to confirm pool initializes)
• Review BREAKING_CHANGES.md for detailed migration guidance
• Test IMC behavior with the new System cache pool to verify performance improvements