plugin: typed session entries + custom renderers — P2 - #39
Closed
yogthos wants to merge 3 commits into
Closed
Conversation
added 3 commits
May 20, 2026 11:53
Preparatory work for P3 (granular streaming hook events) and P4
(session tree branch-aware accounting). Rig's multi_turn stream is
a flat sequence of assistant content + tool results + a final response;
this commit introduces a pure state machine that detects per-turn
boundaries and emits them on the event channel.
A "turn" = one LLM call + the tool calls it dispatched + the tool
results returning. Pure-text response = 1 turn. Run with two cycles
of tool calls = 2 turns.
Implementation:
* `TurnTracker` (src/agent/runner.rs) is a 3-state state machine —
Idle, InTurn, AwaitingNext — that consumes observation calls
(`observe_assistant_content`, `observe_tool_result`,
`observe_stream_end`) and returns `Vec<Boundary>` describing the
TurnStart/TurnEnd events to emit.
* The InTurn→AwaitingNext→InTurn transition collapses N parallel
tool results into a single turn boundary, no matter how many tool
results arrive between assistant messages.
* Lazy emission: TurnEnd fires either when the next assistant
content begins (closing the previous turn) or on stream end. This
avoids needing lookahead to know whether more tool results are
still coming.
* `run_stream` instantiates one TurnTracker and forwards Boundary
events through the existing `event_tx` before each assistant-content
event and at stream termination (Done, Error, Interjected, or
unexpected stream close).
New events:
* `AgentEvent::TurnStart { index: u32 }` — fires before the first
assistant content of each turn.
* `AgentEvent::TurnEnd { index: u32 }` — fires after the last event
of each turn (either before the next TurnStart, or before Done /
Error / Interjected).
The UI loop currently handles both with a no-op match arm; P3 will
wire them into plugin `on-turn-start` / `on-turn-end` hooks.
Tests: 8 new unit tests on `TurnTracker` in `mod turn_tracker_tests` —
pure-text, single tool call, multi-result collapsing, alternating
turns, empty stream, mid-dispatch end, lone tool result, restart
reuse. Total: 510 pass with plugin feature (was 502); 460 pass
without (was 452); 12 pre-existing plugin-test fails unchanged.
Refs dirge-e17.
Builds on P0's turn-boundary detection. Wires the new
AgentEvent::TurnStart / TurnEnd events into three new plugin hooks:
on-turn-start {:index N}
on-message-update {:index N :partial "text-so-far"}
on-turn-end {:index N :message "full turn text"}
These give plugins per-turn observability that wasn't possible from
the previous `on-response` hook alone (which only fires once at the
end of the whole agent run). Typical uses:
* Per-turn cost / latency tracking.
* Token-level filtering or annotation.
* Cancel-on-condition based on streaming output.
* Streaming a live transcript to an external sink.
Implementation:
* `src/ui/streaming.rs` — new TokenBatcher type that collects tokens
since the last flush and yields the accumulated text once the count
threshold (DEFAULT_BATCH_TOKENS = 16) is crossed. Count-based
rather than time-based so the tests are deterministic without
mocking Instant. Module is cfg-gated to the plugin feature.
* `src/ui/mod.rs`:
- New per-turn streaming state (token_batcher, current_turn_text,
current_turn_index), cfg-gated.
- AgentEvent::Token arm now pushes to the batcher; on flush,
dispatches `on-message-update` with the accumulated text.
- AgentEvent::TurnStart arm resets per-turn state and dispatches
`on-turn-start`.
- AgentEvent::TurnEnd arm flushes any trailing partial batch (as
a final `on-message-update`), then dispatches `on-turn-end`
with the full turn text.
* `src/main.rs` — adds the three hook names to auto-discovery.
`harness/replace-message` (mutating the persisted assistant text from
on-message-end) is deferred. It needs session-write position tracking
and interacts with usage accounting; will land in a follow-up phase.
Tests: 6 new unit tests on `TokenBatcher` — threshold yielding,
post-flush fresh batch, partial-batch draining, reset, zero-threshold
clamping, lossless content over a run. Total: 516 pass with plugin
(was 510); 460/12 baseline without plugin unchanged.
plugins/turn_timing.janet — example using on-turn-start / on-turn-end
to notify the user how long each turn took.
Refs dirge-87x.
Plugins can now record typed entries on the session timeline and
register custom renderers to display them. Persists across session
save/load via a `Session::extra_entries` field that defaults on
deserialize (pre-P2 session files load without migration).
New harness APIs:
(harness/append-entry "type" "data" &opt display)
Records a typed entry. `data` is opaque to the host — the
plugin chose its own format (plain text, JSON, etc.) and any
registered renderer for `type` formats it on display.
`display` defaults to true; false-display entries persist
silently (useful for plugin state like telemetry counters
that shouldn't visually clutter the chat).
(harness/register-renderer "type" "fn-name")
Associates a custom_type with a Janet function. The host
calls the function with the entry's data string when
displaying entries of that type.
(harness/render color text)
Called from inside a registered renderer. Emits one line of
output with the given color name. Multiple `harness/render`
calls per renderer invocation produce multiple lines.
Default renderer: when no renderer is registered for an entry's
custom_type (e.g. the producing plugin is uninstalled), the host
falls back to a minimal "[entry: TYPE]" header + dim raw data
dump. Persisted entries from removed plugins still show up.
Color names accepted: black, red, green, yellow, blue, magenta,
cyan, white, plus dark* variants and grey/gray. Unrecognized names
fall back to dim grey.
Data flow:
* Plugin: `(harness/append-entry ...)` writes to a `\type\tdata\tdisplay\n`
blob in Janet. Embedded tabs/newlines/backslashes are escaped via
the harness's `-escape` helper.
* Host: `PluginManager::drain_entries()` reads the blob, unescapes
via `unescape_harness_field`, clears, returns `Vec<(type, data, display)>`.
* UI loop top: drains each tick, calls `Session::append_plugin_entry`
to record (with monotonic seq + timestamp), then renders via the
registered renderer.
Persistence: `Session::extra_entries: Vec<PluginEntry>` with
`#[serde(default)]` so pre-P2 files round-trip unchanged.
Tests: 10 new in plugin::tests, plus `test_unescape_harness_field_roundtrips`
that's non-gated so the escape helper has coverage on both feature
flavors. Total: 526 pass with plugin (was 516); 461 + 12 pre-existing
without (was 460 + 12).
plugins/bookmark.janet — example combining `/bookmark <label>` slash
command + custom cyan-star rendering via append-entry +
register-renderer.
Known limitation (deferred to P2 follow-up): entries render at the
end of the session view rather than interleaved with messages by
timestamp. Interleaving requires per-message timestamps which messages
don't carry today.
Refs dirge-u49.
3 tasks
yogthos
force-pushed
the
feat/plugin-streaming-hooks
branch
from
May 20, 2026 16:45
ff08d5a to
6032415
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Plugins can record typed entries on the session timeline and register
custom renderers to display them. Persists across save/load via a new
`Session::extra_entries` field that defaults on deserialize, so
pre-P2 session files load unchanged.
New harness APIs
```janet
; Record a typed entry. data is opaque; the plugin chose its format.
(harness/append-entry "type" "data" &opt display)
; Register a renderer to format entries of a given type.
(harness/register-renderer "type" "fn-name")
; Inside the renderer, emit one or more colored lines.
(defn fn-name [data]
(harness/render "cyan" (string "★ " data))
(harness/render "white" "details below"))
```
When no renderer is registered for an entry's custom_type (e.g.
because the producing plugin is uninstalled), the host falls back
to a minimal `[entry: TYPE]` header + dim raw-data dump.
display=false entries persist but don't render — useful for
silent plugin state.
Implementation
for backward-compatible session JSONL loading.
`invoke_renderer`. Janet stores entries as a tab-delimited blob
with embedded tab/newline/backslash escaping; host unescapes via
a new `unescape_harness_field` helper.
Janet fn, then parses back `color\ttext\n` lines.
drain), calls `Session::append_plugin_entry`, then renders via
the registered renderer or default fallback.
crossterm palette (black/red/green/yellow/blue/magenta/cyan/white
Known limitation
Entries render at the end of the session view rather than interleaved
with messages by timestamp. Interleaving requires per-message
timestamps which the existing message format doesn't carry today —
deferred to a P2 follow-up.
Test plan
10 new unit tests in plugin::tests:
`plugins/bookmark.janet` is included as a working example:
`/bookmark ` records a bookmark; a registered renderer
shows it as a cyan `★ ` line in the chat.
Stacked on
Builds on PR #38 (P3 streaming hooks). Targets
`feat/plugin-streaming-hooks` so the diff is just the new APIs
Refs dirge-u49.