erllama 0.11.0

Changed

• chat/3 defaults reasoning_format to deepseek: thinking models
  now return their thinking text in reasoning_content instead of
  inline in content (llama-server's default). Opt out per call with
  reasoning_format => none.
• chat_apply/3 returns the template's constraint set alongside
  prompt and params: sampler_opts, stop_sequences,
  generation_prompt, supports_thinking, thinking_start_tag,
  thinking_end_tags. Merge sampler_opts + stop_sequences into
  the stream/3 options (the tool-calls guide shows how).

Added

• erllama:fork_session/3: duplicate a sticky session's live KV into
  a new session (llama_memory_seq_cp), so two continuations explore
  different branches without re-prefilling the shared prefix. Works
  on every model family; never queues (seq_capacity when no
  sequence is free).
• Load progress: progress_to => Pid on load_model delivers
  {erllama_load_progress, ModelId, Float} messages (whole-percent
  throttled, final 1.0) while the GGUF loads.
• Native logs: llama.cpp / ggml log lines are forwarded into logger
  under the domain [erllama, native], gated by the native_log_level
  application env (none | error | warning | info | debug, default
  warning).
• Full llama.cpp sampler surface as request options: typical_p,
  top_n_sigma, xtc_probability / xtc_threshold,
  dynatemp_range / dynatemp_exponent, min_keep,
  frequency_penalty / presence_penalty / penalty_last_n, DRY
  (dry_multiplier, dry_base, dry_allowed_length,
  dry_penalty_last_n, dry_sequence_breakers), mirostat (1 | 2)
  with mirostat_tau / mirostat_eta, logit_bias, ignore_eos
  and infill. Chain order follows llama.cpp's default; see the
  sampling reference in the configuration guide.
• Per-token logprobs: logprobs => N (1..32) emits
  {erllama, Ref, {logprobs, #{token_id, logprob, top}}} stream
  events and adds a logprobs list to complete/3 / collect/2
  results. Full-vocab log-softmax over the raw model distribution
  (OpenAI semantics).
• erllama:vocab_info/1: vocabulary size, add_bos / add_eos, and
  the special / FIM token ids (bos, eos, eot, sep, nl,
  pad, mask, fim_pre, fim_suf, fim_mid, fim_pad,
  fim_rep, fim_sep) for fill-in-the-middle prompt assembly.
• erllama:detokenize/3 with remove_special / unparse_special
  (renders special tokens), backed by llama_detokenize.
  detokenize/2 is untouched: the cache byte-keys are computed over
  its output.
• Tool-call grammar enforcement: chat/3 merges the grammar llama.cpp
  synthesizes from the chat template into the request, so
  tool_choice => required always yields a parsed call and
  tool_choice => auto arms a lazy grammar that constrains the reply
  once the model opens a call. Template-declared stop strings are
  honoured. A caller grammar combined with active tools is rejected.
• json_schema chat option (OpenAI response_format semantics): the
  reply's content is grammar-constrained to the schema. Rejected in
  combination with tools.
• Chat options enable_thinking (default true) and
  continue_final_message (none | auto | content | reasoning) for
  assistant prefill.
• Request options grammar_lazy, trigger_patterns,
  trigger_tokens, grammar_prefill: the lazy / template-grammar
  sampler shape (llama_sampler_init_grammar_lazy_patterns), usable
  directly with stream/3 and friends.

Fixed

• Cache correctness on recurrent / hybrid models (Mamba, RWKV, Jamba,
  Qwen3-Next, LFM2, ...): a warm exact hit no longer duplicates the
  last prompt token when the memory refuses the primer's partial
  removal; the engine falls back to a cold prefill and bumps the new
  restore_failed counter. A failed kv_unpack no longer crashes the
  model process (the row is treated as a miss).
• Partial warm hits (extended prompts) prefill the suffix directly
  instead of dropping and re-decoding the last cached token: one fewer
  decoded token per partial hit, and no partial seq_rm, so the path
  stays warm on recurrent models.
• nif_kv_seq_rm refreshes the per-seq position bookkeeping on
  failure too, keeping Erlang and llama.cpp in agreement when a
  partial removal is refused.

Added

• Model-family probe at load: model_info/1 reports arch,
  n_ctx_train, n_params, n_embd, n_layer, n_swa,
  recurrent and hybrid (llama backend). Encoder-decoder (T5) and
  diffusion archs are rejected at load with
  {error, {unsupported_model, encoder_decoder | diffusion}} instead
  of failing confusingly at the first decode.
• Context option n_rs_seq (recurrent-state rollback snapshots per
  seq); defaults to 1 on recurrent / hybrid models so warm hits work
  where the arch supports rollback.
• split_mode => tensor (experimental upstream tensor-parallel
  split; row is deprecated upstream).
• Cache counter restore_failed; stub backend knobs
  fail_seq_rm_last / fail_kv_unpack to test the fallback paths.

erllama 0.10.1

Fixed

• chat/3 and chat_apply/3 hand tools to llama.cpp in the OpenAI
  shape ({"type": "function", "function": {...}}); the flat
  chat_tool() maps were rejected with chat_parse_failed.
• erllama_chat_SUITE starts the application with its dependencies when
  run on its own.

Added

• Load option chat_template: Jinja source replacing the template
  stored in the GGUF for chat/3 / chat_apply/3.
• examples/agent_loop: a tool-calling agent loop on chat/3.

erllama 0.10.0

Changed

• Vendored llama.cpp bumped from b10068 to b10593 (vendor/ is now kept
  whole: upstream builds it as CMake targets). model_opts gains
  load_mode (upstream's llama_load_mode); use_mmap / use_mlock
  are mapped onto it.

Removed (BREAKING)

• unload_model/1 (use unload/1), models/0 (use
  list_models/0), list_cached_prefixes/2 (renamed
  cached_prefix_len/2).
• infer/4: use stream/3 (text or tokens; the receiving
  process is the to option, default the caller). continue/3 takes
  to instead of caller_pid; a missing session_id is
  {error, {missing_option, session_id}}.
• Stream messages {erllama_token, Ref, _}, {erllama_token_id, Ref, _}, {erllama_thinking_end, Ref, _}, {erllama_done, Ref, _},
  {erllama_error, Ref, _}: every event is now {erllama, Ref, Event} with Event :: {token, Bin} | {token_id, Id} | {thinking, Bin} | {thinking_end, Sig} | {done, Stats} | {error, Reason}
  (erllama:stream_event()).
• apply_chat_template/2 renamed render_chat_template/2.
• chat_apply/2 is chat_apply/3 (model, messages, opts) and
  returns {ok, #{prompt, params}}; messages and tools are Erlang
  maps, JSON encoding happens at the NIF boundary.
• verify/4 returns {ok, #{accepted, next}}.
• set_observer/1 and clear_observer/0: use a
  middleware (erllama_middleware, guides/middleware.md).
• Application environment: chat_params_cache_size renamed
  chat_cache_size; quota_mb dropped from the tiers entries.

Changed (BREAKING)

• Every per-model call returns {ok, Result} or {error, not_loaded}
  for an unknown or stopped model instead of exiting with noproc:
  model_info/1, status/1, phase/1, pending_len/1,
  queue_depth/1, last_cache_hit/1, list_adapters/1 now wrap
  their result in {ok, _}; unload/1, evict/1, shutdown/1,
  end_session/2 return {error, not_loaded}.
• load_model/1,2 validates the config (erllama_opts): backend
  defaults to erllama_model_llama; a missing model_path is
  {error, {missing_config, model_path}}, a missing file is
  {error, {invalid_config, model_path, Path}}, an unknown key is
  {error, {unknown_option, Key}}. model_id in the config map is
  honoured by load_model/1.
• complete/3, prefill_only/3 and infer/4 validate their option
  maps: unknown keys are {error, {unknown_option, Key}}, wrong types
  {error, {invalid_option, Key, Value}}.
• response_tokens defaults to 64 on every path (complete/3 used 4).
• evict/1 and shutdown/1 honour the evict_save_timeout_ms
  application environment key (default 30 s); it was documented but
  unread.
• list_adapters/1 entries use the key adapter (was handle).

Added

• erllama:stream/3 and erllama:collect/2: streaming inference
  with a typed event envelope and a collector that folds the events
  into a stream_result().
• erllama:chat/3: one chat turn (render, generate, parse) with
  Erlang-term messages and tools; returns the parsed assistant
  message with content, reasoning and tool calls.
• erllama:embed/2 accepts text; erllama:embed_batch/2 embeds a
  list of inputs in one round-trip to the model process.
• erllama:whereis/1 returns the model pid for monitoring.
• Supervised cache tiers: erllama_cache:add_tier/1, remove_tier/1,
  list_tiers/0, info/0, and the tiers application environment
  key ([#{name, backend => disk | ram_file, root}]) started with
  the application. load_model checks that tier_srv is running and
  matches tier.
• erllama_middleware: hackney-style middleware chain around every
  API call (global via the middleware environment key, or per call
  with the middleware option).
• erllama:pressure/0, pressure_sources/0, requests/0,
  request_info/1.
• Application environment keys fingerprint_mode (now the default
  for models that do not set it), writer_max_concurrent,
  chat_cache_size, thinking_signing_key, middleware and tiers
  are declared in the app file and documented; os_mon is a
  declared dependency (the system pressure source needs memsup).
• erllama_scheduler:validate_config/1 checks that model_evictor
  names a loadable module exporting evict_one/0.
• Documentation: public modules are erllama, erllama_cache,
  erllama_middleware, erllama_scheduler and the
  erllama_model_backend, erllama_model_evictor, erllama_pressure
  behaviours plus the erllama_model_stub test backend; every other
  module is hidden from hexdocs. Guides rewritten around the public
  API (the tool-calls guide now documents chat/3; the
  tool_call_markers option it described never existed). Public
  types are defined in erllama.
• Tests: shared erllama_test_helpers; no catch Expr left, so the
  suite compiles on OTP 29 without nowarn_deprecated_catch.
• erllama exports the types its specs use (token_id/0,
  cache_key/0, completion_result/0, stats/0, request_opts/0,
  load_config/0, error_reason/0, ...) and documents every error
  reason in error_reason/0.

erllama 0.9.0

Added

• erllama_chat:set_observer/1' / clear_observer/0' hook.
  Lets a separate module (typically the server's metrics module)
  observe wall-time of every chat-NIF call (apply / parse) without the runtime taking a compile-time
  dependency on the metrics module. Registered via persistent_term;
  unset = no-op. Drives the new
  `erllama_chat_*_duration_seconds' Prometheus histograms
  on the server side.
• erllama_app:start/2' enables erlang:system_flag(scheduler_wall_time, true)' at boot so callers
  (the server's metrics module among them) can read per-scheduler
  busy ratios for the autoparser dirty-pool monitoring.

Changed

• chat_apply/2' runs upstream's common_chat_templates_apply' once per
  request (prompt + parser). The per-tools params cache and the
  render-only NIF are gone; only the per-model templates ref is
  cached.
• Vendored llama.cpp is b10068, upstream and unmodified (no local
  patches). scripts/vendor_llama.sh <tag> performs bumps and fails
  if any vendored file differs from the tarball. Unused vendor/
  libraries are pruned and LLAMA_OPENSSL is off (no OpenSSL link).
• Hex package manifest now includes c_src/erllama_chat_nif.{cpp,h}
  and c_src/erllama_resources.h (previous tarballs could not build).
• Bump vendored llama.cpp from b9334 to b9585. No API-breaking changes
  on our touchpoints; brings common/chat* bug fixes (LFM2 reasoning,
  tool-parser unification). UPDATE_LLAMA.md refreshed to document
  the common/ + vendor/ sync the prior procedure left implicit.

Removed (BREAKING)

• Streaming API: {erllama_token, _, {tool_call_delta, _}}' and erllama_tool_call_end' messages, plus the matching
  step result variants. The engine no longer classifies tool-call
  bytes mid-stream. Callers using erllama:infer/4' directly buffer tokens and call erllama:chat_parse/3'
  at done for structured extraction. HTTP wire format on the
  server is unchanged.
• Backend `tool_call_end_is_eos/1' callback.

Changed

• erllama:chat_apply/3' becomes chat_apply/2' (drops the
  ToolsHash' parameter). erllama_chat_cache' shrinks to
  caching only the heavy common_chat_templates_init' ref per model; each request invokes common_chat_templates_apply' fresh because
  the synthesized parser is sensitive to tool_choice' and parallel_tool_calls' (now folded into the NIF Inputs map).

Added

• Public erllama:chat_apply/2 + chat_parse/3 that delegate
  to erllama_chat via the model gen_statem so callers can
  build a chat_params_ref' and parse model output without touching the underlying NIF model resource. Backend gains an optional get_model_ref/1' callback; the stub backend returns
  {error, chat_not_supported}'. chat_purge/1' drops cached entries
  on demand for a given model id.

Fixed

• EOS-bounded tool-call flush: don't capture the EOS token's bytes
  before flushing the span. The previous in-span EogFlag path called
  req_tool_call_emit/3 unconditionally; for Granite / Phi-4 the EOS
  is a special token that detokenizes empty under special=false,
  so the buffer stayed clean in practice, but the path was fragile
  for any future model whose EOS detokenises to visible bytes. Now
  the EOS-end branch skips the emit and calls req_tool_call_end/2
  directly, matching the byte-string-end-marker semantics.

Added

• Complete the chat-autoparser NIF wiring. nif_chat_templates_apply
  now translates an Erlang inputs map (messages + tools JSON
  binaries, optional tool_choice) into common_chat_templates_inputs
  via common_chat_msgs_parse_oaicompat / common_chat_tools_parse_oaicompat,
  calls upstream, and returns the synthesized chat_params_ref plus
  the rendered prompt bytes. nif_chat_parse deserialises the
  per-template PEG arena from the cached params string and dispatches
  to common_chat_parse. The parsed common_chat_msg is marshalled
  to #{role, content, reasoning_content, tool_calls}; tool-call
  arguments come back as raw JSON binaries and are decoded to maps
  at the Erlang facade boundary (erllama_chat:parse/3).
  New erllama_chat_SUITE real-model CT (gated on
  LLAMA_TEST_MODEL) covers init / apply / parse round-trip and a
  partial-then-full streaming case.
• NIF wrapper for llama.cpp's common_chat_* autoparser. Vendors
  common/ + vendor/nlohmann + vendor/cpp-httplib from the
  pinned llama.cpp tree, flips LLAMA_BUILD_COMMON=ON, and links
  llama-common into the NIF .so. Three new entry points
  (nif_chat_templates_init/2, nif_chat_templates_apply/2,
  nif_chat_parse/3) run on dirty CPU; two new resources
  (chat_templates_ref, chat_params_ref) wrap
  common_chat_templates_ptr and common_chat_params. New Erlang
  facade erllama_chat (raw NIF shim) and
  erllama_chat_cache (LRU cache keyed on
  {ModelIdBin, ToolsHash}, with purge/1 for model unload). The
  refactored erllama_resources.h exposes the existing
  C resource type pointers + erllama_model_t to C++
  TUs behind an extern "C" guard. templates_init works
  end-to-end; templates_apply and parse ship as
  {error, not_implemented} placeholders (the Erlang-term marshalling
  lands in the follow-up). Dormant capability: the chat / messages /
  responses handlers do not consume this surface yet; Phase 3.C
  wires them up.
• EOS-bounded tool-call end-marker capture. Models configured with
  tool_call_markers => #{start => Bytes, 'end' => <<"$eos">>} opt
  into the new path: when the scheduler is inside an open tool-call
  span and the model samples a token with EogFlag = 1, the
  accumulated tool_call_bytes buffer is flushed via the existing
  erllama_tool_call_end message before the request
  finishes. Previously the buffer was silently dropped in that
  branch (the only emission site was the byte-string end-marker
  match in erllama_model_llama:map_marker/2). The
  byte-string-end families (Mistral </s>, Qwen </tool_call>,
  DeepSeek <｜tool▁call▁end｜>, Llama 3.1 <|eom_id|>) are
  byte-exact unchanged; the new path is opt-in via the sentinel
  binary <<"$eos">> on the end' key. Backend behaviour gains one optional tool_call_end_is_eos/1callback (the scheduler defaults tofalse` for backends that have not been updated).
  Targets the IBM Granite-3.x and Microsoft Phi-4-mini families
  whose wire shape bounds the call at EOS rather than a
  byte-string end marker; the server-side migrations land in
  follow-up PRs.

Changed

• The native tool-call capture splits spans on a repeated start marker. Families like
  Mistral tekken delimit parallel calls by emitting the start marker again
  ([TOOL_CALLS]n1[ARGS]a1[TOOL_CALLS]n2[ARGS]a2</s>) with no per-call end between
  them; the previous single-span capture concatenated every call into one
  erllama_tool_call_end message, which the server parsed as one garbled
  call. apply_step_results/2 now finalises the current span before opening the next
  when a start marker fires while already in-span, so each call yields its own end
  message and reuses the upstream per-call accumulation. Behaviour-preserving for
  families with explicit per-call end markers (qwen-xml, qwen3-coder, dsml): a start
  never fires while in-span because the end fires first. The stub backend gains an
  optional tool_call_script :: [start | body | end_tok] config so tests can drive
  exact decode-step sequences (the Mistral tekken [start, body, body, start, body, body, end_tok] shape and the qwen-style [start, body, end_tok, start, body, end_tok] regression guard run from the same harness).
• The tool-call end marker now carries the source token's eog flag.
  erllama_model_llama:map_marker/2 was discarding EogFlag on the end
  marker, so a model whose end marker IS the eos token (Mistral tekken uses </s> as
  both the per-span end marker and the assistant turn's eos) kept decoding past the
  close and spammed repeated tool calls under the greedy continuation. Backend step
  results emit {tool_call_end, Eog} instead of the bare atom; the scheduler sets
  Req.finishing = true on eog so the turn ends cleanly. Backend type spec updated
  accordingly.
• erllama_model_stub gains an opt-in step_delay_ms :: non_neg_integer()
  test knob that timer:sleeps for the configured number of milliseconds at the top
  of every step/2 call. Lets server-side concurrency tests deterministically keep a
  holder request in-flight while other requests race for the queue (used by the e2e
  suite's chat_busy_returns_429 to fix a pre-existing CI timing flake where the
  holder's stream could finish before the racing requests arrived). Default 0 is a
  true no-op for the cache / integration tests that already use the stub. Invalid
  configs coerce to 0.

Fixed

• Idle sticky-session seq pins are now reclaimed under seq-pool pressure, so the pool no
  longer permanently exhausts. A completed sticky turn keeps its sequence pinned for warm
  continuation, but those pins were never released (only end_session/2 or model stop
  freed them) - so after n_seq_max distinct sessions, every new session got
  {error, seq_capacity} (529) with retries never recovering. Admission now reclaims the
  least-recently-used idle pin (a session_seq entry whose seq has no in-flight
  request) when the pool is full, on both the fresh-admit (admit_normal/2) and queued-
  dispatch (dispatch_pending_admits/2) paths, while never reclaiming an in-flight
  session. seq_capacity now only fires when every seq is genuinely active. A reclaimed
  session re-admits cold (or warm-restores from the tiered cache) on its next turn.
  model_info/1 gains pinned_idle_seqs (reclaimable headroom; available_seqs stays
  ~0 since an admitted request immediately re-pins).
• Native tool-call capture now keeps the body between the markers. The marker scanner
  (erllama_model_llama:map_marker/2) is stateless - it tags only the start/end
  marke...

v0.8.0

[0.8.0] - 2026-05-23

Engine-robustness release covering the erllama_server hardening brief observed under real 30B/Metal load (cold-admit decode wedges, agentic tool-continue loops).

Added

• Bounded, interruptible, self-recovering decode. Every context installs a ggml abort callback. Each decode step arms a per-step wall-clock budget (context_opts.decode_budget_ms, default 30000, 0 disables); exceeding it returns {error, decode_timeout} instead of blocking. erllama_nif:request_abort/1 sets an atomic flag the callback honours without taking the context mutex, so a running decode can be interrupted from outside the blocked gen_statem and returns {error, decode_aborted}; cancel/1 fires it best-effort. On decode_timeout/decode_aborted the engine recovers in place: fails in-flight and queued callers, recreates the context via the new backend reset_context/1 (model stays loaded), resets seq and session state, returns to idle. Recovery drops only the live in-context KV and sticky-session pins; the persistent tiered cache survives, so the next admission still warm-restores from cache rather than starting fully cold.
• on_full => block | error admission option on complete/3, prefill_only/3, infer/4 (default block). error fails fast with {error, seq_capacity} instead of queueing when no seq is free; pair with available_seqs / n_seq_max from model_info/1.
• generated => [token_id()] in the erllama_done Stats map: the exact generated token ids in order, so a caller can build a byte-exact suffix for continue/3 without re-tokenising detokenised text.
• expect_committed => [token_id()] option on continue/3: the caller's view of the session's committed tokens. When supplied it must equal the stored context exactly, otherwise continue/3 returns {error, {transcript_mismatch, #{stored_len, expected_len, diverge_at}}} without prefilling, leaving the seq pinned for a re-sync and retry.

Changed

• A binary grammar is now authoritative through tool-call syntax tokens. Previously, on a model with tool_call_markers, a tool_choice=required / response_format grammar was abandoned once a tool-call span opened (syntax tokens went through a grammar-less greedy sampler), letting output drift to free-form. The greedy-on-syntax swap is now disabled for any request carrying a grammar, so the constraint holds end to end on infer/4 and continue/3.
• nif_decode_one now returns {error, {decode_failed, Rc}} instead of a bare {error, Rc}, matching nif_step.

v0.7.0

[0.7.0] - 2026-05-20

Added

• erllama:reset_session/2 recovery primitive. Forcibly drops a
  sticky session's live KV cells and any in-flight #req{} on its
  seq, then returns the seq slot to the idle pool. Uses a 5 s
  gen_statem:call timeout so it stays reachable when the engine's
  infinity-timeout hot path is wedged. Returns
  {ok, recovered | not_found} | {error, timeout}. Streaming
  callers on the reset seq receive {erllama_error, Ref, engine_reset}.
• n_seq_max and available_seqs keys in erllama:model_info/1.
  available_seqs is the live idle-list length; sticky-pinned seqs
  count as unavailable. Lets callers detect saturation up front
  instead of inferring it from sticky_busy errors.

Changed

• nif_step now returns {error, {decode_failed, Rc}} (was bare
  {error, decode_failed}). The Rc integer surfaces the
  llama_decode return code (1, -1, 2, ...) so operators can tell
  OOM from KV-cache corruption from sample rejection. The exception
  path ({error, exception}) is unchanged.

[0.6.2] - 2026-05-19

Changed

• Vendored llama.cpp bumped from b9119 to b9222. No erllama
  source changes required; the C ABI surfaces we depend on (model
  load/free, context, sampler chain, llama_n_batch,
  llama_state_seq_*) are unchanged.

v0.6.1

[0.6.1] - 2026-05-18

Fixed

• BEAM segfault in erllama:apply_chat_template/2 on rendered
  chat-template output above the initial 4 KiB render buffer. The
  vendored llama_chat_apply_template returns the full formatted
  size as a positive value even when the caller's buffer was too
  small (strncpy silently truncates). The NIF retry path only
  fired on negative return, so the positive size was fed as
  text_len to llama_tokenize and the subsequent std::string
  construction walked past the truncated buffer into unmapped pages.
  Retry now triggers on written > buf_size. Caps bumped to match
  the downstream's 64 MiB body limit: ERLLAMA_MAX_TOKEN_TEXT
  4 MiB → 64 MiB, ERLLAMA_MAX_TOKENS 1 M → 16 M. The token output
  cap is enforced on the tokenize success path so byte-fallback
  tokenizers don't return over-cap lists.

v0.6.0

[0.6.0] - 2026-05-18

Added

• erllama:continue/3 for caller-asserted chat-template continuation
  on a sticky session. Skips the prompt prefix-equality check in
  resolve_sticky_continuation so chat templates whose rendered
  prefix shifts between turns can still reuse the live KV cells.
  Caller passes the tokenised tail directly; the engine prefills
  only that tail on top of the pinned seq's stored KV. Returns
  {error, no_session} for unknown session ids and
  {error, sticky_busy} for in-flight seqs. New
  cache_hit_kind => continuation reports the path in Stats.
  Companion guide section in guides/examples.md and lifecycle
  notes in internals/request-lifecycle.md.

v0.5.1

[0.5.1] - 2026-05-17

Documentation-only patch on top of 0.5.0.

Added

• ## Tool-call handling section in the README describing what
  erllama exposes (per-model tool_call_markers, the
  {tool_call_delta, _} / {erllama_tool_call_end, _, Full}
  streaming wire, and the automatic greedy-on-syntax sampler
  swap) and what it deliberately leaves to the HTTP layer (tool
  id minting, JSON parsing, canonicalisation).
• New guides/tool-calls.md companion to the README section,
  linked from the documentation table.
• New internals/request-lifecycle.md describing the per-model
  gen_statem admission, cache resolution, decode, and save
  pipeline.
• internals/c-safety-audit.md added to the HexDocs navigation.

Changed

• README rewritten for sharper top-of-funnel: tighter "Why" list,
  cleaner Quick taste, Common patterns block replacing the old
  long example.
• Architecture diagram corrected — erllama_cache_ramfile_srv
  and erllama_cache_disk_srv are operator-started standalone
  servers, not children of erllama_cache_sup. Added
  erllama_registry and erllama_inflight which were missing.
• "Inside a request" lifecycle updated for the multi-seq
  scheduler: two states (idle/running) instead of the v0.1
  three-phase model, with co-batched nif_step and inline
  thinking/tool-call marker recognition.

v0.5.0

[0.5.0] - 2026-05-16

Tool-call exact-replay scaffolding for downstream HTTP front ends.
Models loaded with tool_call_markers produce structured boundary
messages on the streaming wire so a caller can capture the exact
bytes the model sampled, store them under a tool id, and splice
them back verbatim on later turns to keep the KV-cache prefix
match working. Companion primitives expose explicit suffix replay
and sticky per-session seq_id pinning.

Added

• tool_call_markers => #{start, end, payload_start (optional), payload_end (optional)} on erllama:load_model/2 Config. Each
  binary is tokenised through the model's own vocabulary at load
  time; multi-token markers are supported. Omitting the key keeps
  the backend on the existing path (#39, #42).
• Streaming wire on infer/4 gains
  {erllama_token, Ref, {tool_call_delta, Bin}} per chunk and a
  single {erllama_tool_call_end, Ref, Full :: binary()} per
  span, with Full carrying every emitted delta concatenated so
  the downstream's exact-replay map stores them verbatim without
  re-buffering (#39).
• erllama:prefill_only/3 accepting Opts with parent_key.
  When passed a prior turn's finish_key, the call warm-restores
  from that row and prefills only the new suffix before firing the
  finish save — useful for chaining cache-warming calls across
  turns (#40).
• session_id => term() on infer/4 Params and
  complete/3 / prefill_only/3 Opts. Pins the underlying seq_id
  to that session across requests so the next turn whose prompt
  continues the stored tokens truncates-and-prefills in place on
  the already-live KV cells (cache_hit_kind => sticky).
  Concurrent admits on the same session_id return {error, sticky_busy}. Release with erllama:end_session/2 (#41).
• Per-request greedy sampler swap on tool-call syntax tokens.
  Models with tool_call_markers build a second sampler chain
  (temperature => 0) at admission; the scheduler routes syntax
  tokens through it so a tool call is byte-deterministic from a
  fixed prefix. Optional payload markers flip back to the request's
  normal sampler for caller-supplied string contents so they stay
  diverse (#42).

Changed

• step_result() on erllama_model_backend gains four variants
  ({tool_call_token, _}, tool_call_end, {tool_call_payload_open, _}, {tool_call_payload_close, _}). Backends without
  tool-call markers emit none of them.
• erllama_model_stub phase machine re-keyed from sampler ref onto
  seq_id so mid-request sampler swaps (the new greedy-on-syntax
  path) don't reset state across ticks. seq_rm now cleans the
  per-seq phase entry.