Skip to content

v0.23.0rc1 — wasm ca4dce224b1d6f028ec37592e0ac504fbf466a7ea23e57c20c7d047dfcaedd0e

Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 22 Oct 00:04
· 19 commits to main since this release

Overview

v0.23.0rc1 is the first release candidate of v0.23.0, the self-call release: one update call can now return a whole answer, because the canister yields to itself between chunks. run_update_full generates a complete response, run_decision_full answers a whole decision request, and run_update stops before the instruction limit instead of trapping. It also keeps the last logits next to each prompt cache, makes the cap on stored Kev states per scope configurable, adds a log monitor, and moves to icpp-pro 7.0.0rc2.

Highlights

  • New run_update_full: ingests the prompt and generates the whole response in one update call, yielding to itself between chunks (#54)
  • New run_decision_full: answers a whole decision request in one update call, for Julia-1 and Kev-0.8B (#55)
  • run_update stops a chunk at a 36B instruction guard instead of trapping at the 40B limit, so a longer context cannot trap (#53)
  • A continued session samples its first token from its own logits, saved in a .logits sidecar, never from another caller's (#50)
  • Every README uses one run_update_full or run_decision_full call, with max_tokens_update = 0 (no cap) (#56, #60)

Behaviour changes for apps

  • A run_update chunk can end before max_tokens_update tokens at the instruction guard; keep looping until the response ends (#53)
  • run_update_full needs --prompt-cache, refuses --model (400), and sets partial = true when stopped or after 500 chunks (#54)
  • A session (principal plus prompt cache) held by a run_update_full gets 409 from run_update, new_chat and remove_prompt_cache (#54)
  • A held session expires after 10 minutes without progress; copy_prompt_cache and upload_prompt_cache_chunk to it return an error (#54)
  • While a run_update_full or run_decision_full runs, load_model and run_update --model return 409 (#54, #55)
  • Laya: run_decision_full takes one question per request; a step after a yield returns an Err that points to run_decision (#55, #60)
  • instructions in a decision reply now counts every message of the call, for run_decision too (#55)
  • At max_tokens_update = 0, a multi-question Julia-1 or Laya request no longer traps: first-message passes are guarded at 36B (#58)
  • Laya's first question after load_model is not checked: a too-long one gets an IC0522 reject; set max_tokens_update = 32 for an Err (#58, #60)

Candid API changes

  • New run_update_full : (InputRecord) -> (OutputRecordResult) and run_decision_full : (DecisionInputRecord) -> (DecisionResult) (#54, #55)
  • RunOutputRecord gains partial : opt bool, set only by run_update_full (#54)
  • max_decision_states_per_scope : opt nat64 in the cache-cleanup config input, config record and stats record (#59)

Upgrade notes

  • Prompt caches written by an older build still load; without a .logits sidecar they continue as before and get one after their next call (#50)
  • max_decision_states_per_scope (default 8, 0 = no limit) is saved in .canister_settings.json and survives upgrades (#59)

Models and guides

  • Kev-0.8B guide: a walkthrough with one run_decision_full call per request, "Ask one question at a time", and a new demo GIF (#56)
  • Decision guides use run_decision_full, document its partial reply and the cap per scope; Laya sends one question per request (#56, #59, #60)
  • Chat guides (Qwen2.5, Gemma-3, Qwen3-1.7B, LFM2.5-1.2B, Hindi) use one run_update_full call, with measured outputs (#60)
  • The Hindi guide recommends Gemma-3-1B-it with one run_update_full call (#60)
  • LFM2.5-2.6B: the file now on Hugging Face does not load on a fresh canister with this build (2 GiB stable-read limit) (#60)

llama.cpp fork

  • Fork commit 702142008 adds llama_set_logits_last to restore saved logits after a state load; same upstream base, b11476 (#50)

icpp-pro

  • icpp-pro 7.0.0rc2 (from 6.3.2), whose yield_fresh_budget the self-call endpoints use (#47, #51)

Scripts and tooling

  • New scripts/monitor_logs.py (wrapper monitor_logs.sh): saves the canister log outside the canister, without losing or repeating lines (#57)
  • The README recommends raising the canister log memory to 2 MiB, so busy refusals and traps are not rotated out (#57)
  • A release tag with rc is published as a GitHub prerelease, so it never becomes "Latest" (#61)

Changelog since v0.22.0

  • 3e33a44 Release a tag with rc as a prerelease, and add the rc and final paths to the release skill (#61)
  • c566d3b Add scripts/monitor_logs.py: save the canister log outside the canister, without losing or repeating lines (#57)
  • b0db111 Use run_update_full and run_decision_full in every README example (#60)
  • 0340683 Add max_decision_states_per_scope: a configurable cap on stored kev states per scope (#59)
  • ffed655 Guard laya passes in the first message, so max_tokens_update = 0 never traps (#58)
  • 5fa3091 Document run_update_full and run_decision_full, with the Kev walkthrough and a new demo GIF (#56)
  • c47df88 Add run_decision_full: the whole request in one update call, yielding to itself between steps (#55)
  • ba6fb1c Add run_update_full: the whole response in one update call, yielding to itself between chunks (#54)
  • 13fe9e7 Stop run_update chunks on instructions (36B guard) instead of trapping at 40B (#53)
  • 0cc2e8f Split run() and run_decision() into from_wire, chunk/step and reply (#52)
  • c92fbeb Bump icpp-pro to 7.0.0rc2 (#51)
  • e92d637 Persist the last logits in a .logits sidecar so a continued session never samples from another caller's logits (#50)
  • 854c289 Use IC_API::performance_counter instead of a hand-declared ic0 import (#48)
  • 3350506 Bump icpp-pro to 7.0.0rc1 (#47)

Reproducible build

llama_cpp.wasm sha256 ca4dce224b1d6f028ec37592e0ac504fbf466a7ea23e57c20c7d047dfcaedd0e
llama_cpp_canister commit 98a8d4a6c6ec58c8bc340c4329d019c7db94d810
llama_cpp_onicai_fork commit 702142008552e33844f99dddefc409550166d175
built with Docker, linux/amd64, icpp-pro 7.0.0rc2
built on 2026-10-22

Reproduce this wasm yourself

git clone https://github.com/onicai/llama_cpp_canister.git
cd llama_cpp_canister
git checkout 98a8d4a6c6ec58c8bc340c4329d019c7db94d810
make docker-build-base
make docker-build-wasm

The last command prints the sha256. It must equal
ca4dce224b1d6f028ec37592e0ac504fbf466a7ea23e57c20c7d047dfcaedd0e.

No Mac and no local toolchain are needed: the llama.cpp fork commit,
icpp-pro, wasi-sdk, Rust and Binaryen are all pinned inside the image.

Check what a deployed canister is running

Anyone can read a deployed module hash, with no special rights, from the
public read-state path:

dfx canister --network ic info <canister-id>

It must print Module hash: 0xca4dce224b1d6f028ec37592e0ac504fbf466a7ea23e57c20c7d047dfcaedd0e.

If you control the canister you can instead build and compare in one step:

make docker-verify-wasm VERIFY_CANISTER=<canister-id>

(That target uses icp canister status, a controller-only management
call; it fails with IC0542 for anyone else.)