Skip to content

v1.31.3

Choose a tag to compare

View all tags
@ardan-bkennedy ardan-bkennedy released this 17 Aug 00:52
· 32 commits to main since this release
v1.31.3
6884f62
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Release Notes - v1.31.3

Release Date: August 16, 2026

Overview

Version 1.31.3 delivers a major architectural reorganization of the speculative decoding engine, separating it into well-defined packages for classic and MTP (Multi-Token Prediction) strategies. This refactor significantly improves code maintainability and readability while also fixing validation bugs in chat processing.

Detailed Changes

New Features

Speculative Decoding Package Refactor

  • Author: William Kennedy
  • Separated speculative decoding logic from the batch engine into dedicated packages under sdk/kronk/model/internal/speculation/
  • Created classic package implementing classic speculative decoding with generate, verify, and state management components
  • Created mtp package implementing MTP-specific draft generation, synchronization, verification, and finalization
  • Introduced speculation.Controller interface with Disabled fallback implementation for non-speculative workloads
  • Added speculation_runtime.go as the central runtime controller that dispatches to classic or MTP engines based on configuration
  • Added speculation_config.go to centralize speculation configuration parsing and validation
  • Added mtp_runtime_hooks.go with MTP-specific runtime hooks for the yzma integration
  • All new packages include comprehensive test coverage (disabled_test.go, classic_test.go, mtp/*_test.go, speculation_test.go)

Improvements

Validation Bug Fixes

  • Author: William Kennedy
  • Fixed validation bugs in speculative decoding configuration and chat processing
  • Added 57 new lines of tests in cmd/server/api/services/kronk/tests/chat_test.go for chat validation scenarios
  • Added 75 new lines of tests in sdk/kronk/model/chat_test.go for chat processing validation
  • Streamlined batch engine code by removing speculative decoding logic (270 lines removed from batchgen_engine.go, 748 lines from batchgen_speculative.go)
  • Cleaned up batchgen_slot.go and batchgen_slot_start.go to remove speculative decoding concerns
  • Removed speculative_sparse.go (109 lines) as the functionality has been reorganized into the new speculation packages

Documentation

Speculative Decoding Documentation Updates

  • Author: William Kennedy
  • Updated .manual/chapter-03-model-configuration.md with 12 new lines covering speculation configuration
  • Updated .manual/chapter-06-speculative-decoding-mtp.md with 6 new lines reflecting the new architecture
  • Updated BUI components (DocsManual.tsx, DocsSDKModel.tsx) with 42 new lines of documentation content

Dependencies

Configuration Tool Updates

  • Updated sdk/tools/models/kronkconfig.go and kronkresolve.go to support new speculation configuration fields
  • Added tests for kronkresolve configuration parsing
  • Updated model config YAML files in sdk/tools/defaults/yaml/ and zarf/kms/

Statistics

Category Count
Total Commits 2
New Features 1
Improvements 1
Bug Fixes 1
Documentation 1

Contributors:

  • William Kennedy

Upgrade Notes

Breaking Changes

None

Migration

No migration required. The speculative decoding functionality remains the same from a user perspective — the changes are internal reorganization of the codebase.

Recommended Actions

  • No action required for existing users
  • If you have custom speculation configuration, verify it still works with the updated configuration parsing
  • Review the updated documentation in Chapter 3 and Chapter 6 for the latest speculation configuration guidance
Assets 15
Loading