v0.2.83 — Reasoning latency metrics and a leaner chat path

v0.2.83 — Reasoning latency metrics and a leaner chat path

This release adds reasoning-aware time-to-first-token metrics, so you can see what you actually wait for on a reasoning model — the "thinking" stream that starts before the first answer token — plus a send-relative breakdown that exposes the backend "setup before model" overhead the old action-relative metric missed. It then cuts that overhead with a consolidated governance round-trip and pre-stream parallelization, enforces role-scoped feature flags in chat, and records usage, audit, and metrics for content-policy-blocked turns.

🔒 Security

• Role-scoped feature flags are now enforced in chat. The resolver was invoked without the user's role, so scope: 'role' rules silently never matched even though the UI reported them active. The role already resolved for the budget check is threaded through, so role-targeted flags apply consistently on the backend. (#1837)

🚀 Features

• Reasoning-aware time-to-first-token metrics: the message info dialog now shows "Time to first reasoning" — the wait until a reasoning model's thinking starts streaming, which happens before the first answer token — and a send-relative "Setup before model" breakdown that captures the backend pre-stream hops the previous action-relative metric never measured. New optional timeToFirstReasoningMs and timeFromSendMs message-metadata fields back the dialog. Includes a dev/admin-gated "direct TTFT" probe that streams straight to the model — bypassing the persistence/delta/scheduler layers — so a developer can compare the pipeline against the model's own floor. (#1837)

⚡ Performance

• Faster chat "Setup before model" wait: the per-turn governance reads (default-model override, accessible-model filter, model-access check, plus org membership and team IDs) are folded into a single resolveGenerationGovernance round-trip, and the pre-resolved role/team context is threaded through the generation path so startChat and budget enforcement skip duplicate Better Auth lookups. Two cross-component getThread calls are replaced with a direct metadata read. Warm pinned-model path: startChat ~300ms → ~90–200ms; "Setup before model" ~404ms → ~265–290ms. (#1837)
• Pre-stream parallelization: the per-turn skill snapshot is cached in-process (mtime-validated) instead of rebuilt every send via nested node-action round-trips and disk reads, the budget check and feature-flag resolution now run concurrently, and generation is dispatched via an awaited runAction so it can start on a free event-loop tick. (#1837)

🐛 Fixes

• Content-policy ("guardrails") blocks now record a usage-ledger row, an AI audit entry, and timing metrics. The block sites returned early without running onAgentComplete, so a block consumed provider tokens but left no usage, audit, or metric trace. (#1837)
• Overlapping sends no longer double-generate and double-bill: a new send (multi-tab, API, or cancel-then-resend) now cancels the in-flight turn before starting a new one. (#1837)
• Restored the arena first-message branch link, which a content-policy/scheduler split had caused to be created before the user message was saved (dropping the A↔B link). It's now created after the message commits, best-effort. (#1837)
• Restored synchronous PROJECT_MISMATCH detection after generation moved async, mapped all PROJECT_* codes to their localized toast, and wrapped the model-denied audit write so an audit failure can't swallow the user notice. (#1837)

Upgrade

Run tale upgrade to update the CLI, then tale deploy to apply the new version.

No manual migration is required. The two new message-metadata fields (timeToFirstReasoningMs, timeFromSendMs) are optional and apply automatically on deploy; rows written before this release simply omit them. The dev-only direct-TTFT probe reuses the existing TALE_DEPLOYMENT_CONFIG_ADMINS allowlist (empty/unset = nobody, fail-safe) — no new environment variable is needed.

Contributors

@larryro

---

Full Changelog: v0.2.82...v0.2.83