Repository navigation
v0.23.0rc1 — wasm ca4dce224b1d6f028ec37592e0ac504fbf466a7ea23e57c20c7d047dfcaedd0e
Pre-releaseOverview
v0.23.0rc1 is the first release candidate of v0.23.0, the self-call release: one update call can now return a whole answer, because the canister yields to itself between chunks. run_update_full generates a complete response, run_decision_full answers a whole decision request, and run_update stops before the instruction limit instead of trapping. It also keeps the last logits next to each prompt cache, makes the cap on stored Kev states per scope configurable, adds a log monitor, and moves to icpp-pro 7.0.0rc2.
Highlights
- New
run_update_full: ingests the prompt and generates the whole response in one update call, yielding to itself between chunks (#54) - New
run_decision_full: answers a whole decision request in one update call, for Julia-1 and Kev-0.8B (#55) run_updatestops a chunk at a 36B instruction guard instead of trapping at the 40B limit, so a longer context cannot trap (#53)- A continued session samples its first token from its own logits, saved in a
.logitssidecar, never from another caller's (#50) - Every README uses one
run_update_fullorrun_decision_fullcall, withmax_tokens_update = 0(no cap) (#56, #60)
Behaviour changes for apps
- A
run_updatechunk can end beforemax_tokens_updatetokens at the instruction guard; keep looping until the response ends (#53) run_update_fullneeds--prompt-cache, refuses--model(400), and setspartial = truewhen stopped or after 500 chunks (#54)- A session (principal plus prompt cache) held by a
run_update_fullgets 409 fromrun_update,new_chatandremove_prompt_cache(#54) - A held session expires after 10 minutes without progress;
copy_prompt_cacheandupload_prompt_cache_chunkto it return an error (#54) - While a
run_update_fullorrun_decision_fullruns,load_modelandrun_update --modelreturn 409 (#54, #55) - Laya:
run_decision_fulltakes one question per request; a step after a yield returns an Err that points torun_decision(#55, #60) instructionsin a decision reply now counts every message of the call, forrun_decisiontoo (#55)- At
max_tokens_update = 0, a multi-question Julia-1 or Laya request no longer traps: first-message passes are guarded at 36B (#58) - Laya's first question after
load_modelis not checked: a too-long one gets an IC0522 reject; setmax_tokens_update = 32for an Err (#58, #60)
Candid API changes
- New
run_update_full : (InputRecord) -> (OutputRecordResult)andrun_decision_full : (DecisionInputRecord) -> (DecisionResult)(#54, #55) RunOutputRecordgainspartial : opt bool, set only byrun_update_full(#54)max_decision_states_per_scope : opt nat64in the cache-cleanup config input, config record and stats record (#59)
Upgrade notes
- Prompt caches written by an older build still load; without a
.logitssidecar they continue as before and get one after their next call (#50) max_decision_states_per_scope(default 8, 0 = no limit) is saved in.canister_settings.jsonand survives upgrades (#59)
Models and guides
- Kev-0.8B guide: a walkthrough with one
run_decision_fullcall per request, "Ask one question at a time", and a new demo GIF (#56) - Decision guides use
run_decision_full, document its partial reply and the cap per scope; Laya sends one question per request (#56, #59, #60) - Chat guides (Qwen2.5, Gemma-3, Qwen3-1.7B, LFM2.5-1.2B, Hindi) use one
run_update_fullcall, with measured outputs (#60) - The Hindi guide recommends Gemma-3-1B-it with one
run_update_fullcall (#60) - LFM2.5-2.6B: the file now on Hugging Face does not load on a fresh canister with this build (2 GiB stable-read limit) (#60)
llama.cpp fork
- Fork commit
702142008addsllama_set_logits_lastto restore saved logits after a state load; same upstream base, b11476 (#50)
icpp-pro
Scripts and tooling
- New
scripts/monitor_logs.py(wrappermonitor_logs.sh): saves the canister log outside the canister, without losing or repeating lines (#57) - The README recommends raising the canister log memory to 2 MiB, so busy refusals and traps are not rotated out (#57)
- A release tag with
rcis published as a GitHub prerelease, so it never becomes "Latest" (#61)
Changelog since v0.22.0
3e33a44Release a tag with rc as a prerelease, and add the rc and final paths to the release skill (#61)c566d3bAdd scripts/monitor_logs.py: save the canister log outside the canister, without losing or repeating lines (#57)b0db111Use run_update_full and run_decision_full in every README example (#60)0340683Add max_decision_states_per_scope: a configurable cap on stored kev states per scope (#59)ffed655Guard laya passes in the first message, so max_tokens_update = 0 never traps (#58)5fa3091Document run_update_full and run_decision_full, with the Kev walkthrough and a new demo GIF (#56)c47df88Add run_decision_full: the whole request in one update call, yielding to itself between steps (#55)ba6fb1cAdd run_update_full: the whole response in one update call, yielding to itself between chunks (#54)13fe9e7Stop run_update chunks on instructions (36B guard) instead of trapping at 40B (#53)0cc2e8fSplit run() and run_decision() into from_wire, chunk/step and reply (#52)c92fbebBump icpp-pro to 7.0.0rc2 (#51)e92d637Persist the last logits in a .logits sidecar so a continued session never samples from another caller's logits (#50)854c289Use IC_API::performance_counter instead of a hand-declared ic0 import (#48)3350506Bump icpp-pro to 7.0.0rc1 (#47)
Reproducible build
llama_cpp.wasm sha256 |
ca4dce224b1d6f028ec37592e0ac504fbf466a7ea23e57c20c7d047dfcaedd0e |
| llama_cpp_canister commit | 98a8d4a6c6ec58c8bc340c4329d019c7db94d810 |
| llama_cpp_onicai_fork commit | 702142008552e33844f99dddefc409550166d175 |
| built with | Docker, linux/amd64, icpp-pro 7.0.0rc2 |
| built on | 2026-10-22 |
Reproduce this wasm yourself
git clone https://github.com/onicai/llama_cpp_canister.git
cd llama_cpp_canister
git checkout 98a8d4a6c6ec58c8bc340c4329d019c7db94d810
make docker-build-base
make docker-build-wasmThe last command prints the sha256. It must equal
ca4dce224b1d6f028ec37592e0ac504fbf466a7ea23e57c20c7d047dfcaedd0e.
No Mac and no local toolchain are needed: the llama.cpp fork commit,
icpp-pro, wasi-sdk, Rust and Binaryen are all pinned inside the image.
Check what a deployed canister is running
Anyone can read a deployed module hash, with no special rights, from the
public read-state path:
dfx canister --network ic info <canister-id>It must print Module hash: 0xca4dce224b1d6f028ec37592e0ac504fbf466a7ea23e57c20c7d047dfcaedd0e.
If you control the canister you can instead build and compare in one step:
make docker-verify-wasm VERIFY_CANISTER=<canister-id>(That target uses icp canister status, a controller-only management
call; it fails with IC0542 for anyone else.)