Skip to content

v0.4.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 10 Oct 16:26
· 108 commits to main since this release
v0.4.0
80716ae
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

[0.4.0] - 2026-10-09

"Usage ledger": the server keeps one durable, append-only row per model call with its own receive time, answers grouped audit questions and paginated call listings over it, exports it as CSV or JSONL with per-currency reconciliation totals, redacts secrets at export, bounds how long it (and raw events) are kept, and no longer accepts a token in a URL or starts open by accident. Also ships the portal loading full history on open and OTLP metrics ingestion. The one breaking change is listed first; see docs/migrating-to-0.4.md for the full upgrade notes, docs/usage-ledger.md, docs/usage-export.md, docs/retention.md and docs/redaction.md for the references themselves.

Breaking changes

  • Removed: a token or api_key query parameter never authenticates /api/v1 or /v1/logs, in any mode, even when the value is correct (#71), including POST /api/v1/webhooks/generic when no webhook secret is configured. Such a request now gets 401 query_token_not_supported. EventSource clients (which cannot send an Authorization header) now call the new POST /api/v1/stream-tickets first and open the stream with the single-use ticket it returns (?ticket=); connectEventStream in @warlockcode/agent-viewer does this automatically wherever fetch cannot stream a response body. Migration notes, before/after for every client: docs/migrating-to-0.4.md.
  • The server refuses to bind a non-loopback interface with no token (#71). npm run server and a direct startServer() call now default to 127.0.0.1 (AGENT_VIEWER_HOST), not every interface. Setting AGENT_VIEWER_HOST to anything else (for example 0.0.0.0) with no AGENT_VIEWER_API_TOKEN configured makes the process print one line to stderr and exit with code 1 (OpenApiRefusedError, code: 'open_api_refused'). Set a token, bind loopback, or set AGENT_VIEWER_ALLOW_OPEN=1 to accept an open API anyway. The agent-viewer CLI and the Docker images are unaffected: they always run with a token.
  • With no token configured and no AGENT_VIEWER_ALLOW_OPEN=1, /api/v1 now rejects requests that do not look local (#71). A request from a non-loopback remote address gets 403 open_api_loopback_only; a non-loopback Host header gets 403 open_api_host_not_allowed; an Origin header whose host is neither loopback nor listed in an explicit AGENT_VIEWER_CORS_ORIGIN gets 403 open_api_origin_not_allowed (a bare * does not count as a list). A request with no Origin header (curl, the SDKs, the CLI) is unaffected, and an HMAC-signed webhook still works regardless of where it comes from. A reverse proxy in front of the server makes every request look local, so a proxied deployment still needs a token.

Added

  • Crew walk frame pipeline (#117): four original CEO prototype atlases with eight illustrated frames per direction, a versioned manifest and safe prototype-only importer, lazy scene/facing loading, camera transition preload, measured foot anchors and cancellation/reduced-motion/fallback handling in the independent renderer. Playback follows reported isWalking without creating events, usage or navigation. Includes integrity/runtime tests and four-view/mobile/zoom PNG and MP4 evidence; final art and physical paths remain outside this pipeline pilot.

  • Crew visual style contract (#116): a documentation-only deliverable formalizing the "Office Beans" chibi style that the approved CEO reference already draws, so a future illustrator or engineer has one contract to target instead of reverse-engineering the CEO prototype. No art was added or changed. See docs/crew/visual-style-contract.md (English) and its Spanish twin docs/crew/visual-style-contract.es.md.

    • Names the asset lifecycle the issue asked for (reference | vector-study | prototype | approved), ties each state to what asset-manifest.json and scripts/crew-assets.mjs already enforce today, and records that approved and a formal vector-study manifest entry are not reachable yet (follow-up work, out of scope here).
    • Documents the measured character canvas contract (256x352 px RGBA, anchor 0.5/0.9375) that the 11 existing character prototypes already share, the six roles' silhouette/palette cards (CEO's palette authored on the reference sheet, the other five roles' measured automatically from their prototype art and flagged as unratified), the clip/animation contract (ceo.blink.front.v1 as the reference implementation), the ADR-001 layer order and zoom range, and the explicit prohibition on faking a missing facing by stretching or mirroring existing art.
    • scripts/crew-assets.mjs: validateCrewAssets() now also enforces the documented character canvas contract (CREW_CHARACTER_CANVAS 256x352, CREW_CHARACTER_ANCHOR 0.5/0.9375) for every kind: 'character' entry, not just the CEO's four facings. Covered by two new cases in tests/crew-assets.test.mjs plus a dedicated assertion across all 11 character prototypes.

  • Crew renderer split into independent, tested layers (#167): src/crew/renderCrewRoom.ts used to mix room structure, furniture, actor sprites and the presence badge in one function. It is now the orchestrator only; the drawing itself lives in five new modules, each with its own Vitest suite and no import of the other: crewLightingLayer.ts (the shared shading palette: wall/floor tones and per-prop-type face colors), crewSceneLayer.ts (background, walls, floor grid and door thresholds), crewPropsLayer.ts (furniture boxes), crewSpriteLayer.ts (the actor's pose or blink frame) and crewHudLayer.ts (the presence number badge, the only canvas-drawn HUD element). crewRenderPrimitives.ts holds the one shared drawing primitive (a filled, stroked polygon). renderCrewRoom.test.ts and crewViewport.test.ts pass unmodified, confirming identical pixel output and draw order after the split.

    • New crewSceneLifecycle.ts: a framework-independent mountCrewScene() that schedules a render frame with requestAnimationFrame, observes canvas size with ResizeObserver, and exposes update()/unmount(), so a Crew scene's full lifecycle (mount, update, unmount) is unit-testable without React or a real browser (tests/lib/crewSceneLifecycle.test.ts, including an end-to-end test that mounts, switches room and unmounts a real renderCrewRoom scene). It is not yet wired into CrewStage.tsx; that integration is left for a follow-up change, out of this issue's scope, after an attempt to wire it surfaced a regression in crew-sprites.spec.ts that this PR reverts before merging.

  • Two-mode architecture contract: Cartoon (legacy) and Crew (#166): a new ADR-002 formalizes the contract that visualMode: 'cartoon' | 'crew' already implemented across several merged PRs: cartoon stays the default everywhere, switching the mode never recreates the domain OfficeStore, never drops an event and never resets the host's usage/task state, and each renderer's own Crew presentation state (room, per-room camera, reducedMotion) survives the switch whether or not the host controls it. Two new automated checks enforce the contract instead of only documenting it: tests/crew-mode-boundary-166.test.mjs scans the real source tree so Cartoon can never import from src/crew/ and Crew can never import Cartoon's renderer, legacy grid or motion modules (and confirms renderFurnitureItem stays private); tests/lib/crewModeContract166.test.tsx switches visualMode twenty times (the issue's own acceptance threshold) against a growing event stream and asserts the office timer is never duplicated, the latest event is never lost, and Crew's room/camera/reducedMotion state (controlled or not, across two independent AgentOffice instances) never leaks or resets. See ISSUE-166-ACCEPTANCE.md for the full acceptance matrix.

  • Crew spatial model: collisions, functional doors and walkable zones (#170, CREW-005, building on the eleven-room catalog from #168): pure logic over the existing CrewRoomDefinition data, no new art, no new props.

    • src/crew/crewCollision.ts: axis-aligned bounding box (AABB) collision over room data that already existed. propAABB/roomBoundsAABB/agentAABB turn a furniture anchor, a room's floor and an agent's presence radius into boxes; aabbsOverlap, aabbContains and pointInAABB are the primitive tests; crewAgentCollides rejects a spot that leaves the room or overlaps any furniture footprint (with an excludeFurnitureId escape hatch so an agent can re-check its own current anchor); overlappingFurniturePairs is a data-integrity check confirmed empty for all eleven rooms; doorAABB gives the threshold box of a door on the south wall.
    • src/crew/crewModel.ts: each room's doors now also carries connectsTo (the neighbouring room id, or null) and edgeId, built from a new ten-edge spanning tree (CREW_ROOM_EDGES) that connects all eleven rooms into one building, rooted at reception. Every room's original door keeps its exact previous id and position; rooms with more than one neighbour (reception, development, lounge, coffee, meeting, research, qa) get additional doors at new, furniture-clear offsets. No room's furniture, dimensions or rendering changed.
    • src/crew/crewRoutes.ts: the reusable connectivity layer. buildCrewRoomGraph turns the door data into a room adjacency graph; defaultCrewDoorStates/setCrewDoorState/isCrewEdgeOpen are a pure, immutable open/locked state per edge (every door starts open); findCrewRoomRoute and reachableCrewRooms do a breadth-first search across rooms that skips any locked door, so a locked door can fully separate two rooms, exactly as the issue asked. buildCrewWalkGraph turns a room's free presence grid (crewPresenceSlots) into a navigable node graph (4-connected, furniture-aware, with a nearest-neighbour fallback so an off-grid doorway arrival is never an island); findCrewWalkPath is a breadth-first path over it. Both graphs are plain data other systems, such as the future issue #129 agent navigation, can consume directly without recomputing collisions.
    • New tests: tests/lib/crewCollision.test.ts (AABB primitives, the eleven-room furniture-overlap check, bounds/furniture collision across several rooms) and tests/lib/crewRoutes.test.ts (room graph construction, full-building reachability, direct and multi-hop routes, locking a door that disconnects a leaf room or splits the building tree, and walkable-zone connectivity/pathfinding for several rooms).
    • Not in scope for this issue, per its own text: final room art, animated locomotion through a door, and the #129 navigation system itself, which can now be built on top of crewRoutes.ts.

  • Crew renderer performance benchmark suite (#158): no formal FPS/memory benchmark existed for the Crew renderer before this; only an incidental single-canvas check from issue #159. npm run crew:perf:bench measures the pure JS cost of renderCrewRoom for all eleven catalog rooms at four agent tiers (0/6/20/50) with a no-op 2D context, writing docs/crew/perf/latest.json. tests/lib/crewPerfBudget.test.ts turns the same measurement into a CI regression gate (proposed budget: 8ms median at 50 agents, a quarter of the 30 FPS floor). tests/lib/crewMountLeak.test.tsx mounts and unmounts CrewStage 100 times across every room with fake timers, asserting no requestAnimationFrame or window/document listener is ever left behind and at most one <canvas> exists at a time. npm run crew:perf:browser is an informational Playwright script that captures real painted-frame counts and JS heap growth in Chromium (not a CI gate: performance.memory is a non-standard approximation). No new art or scenes were created; methodology, caveats and the latest captured numbers are in docs/crew/PERFORMANCE.md.

  • CSV and JSONL export of the usage ledger, with per-currency reconciliation totals (#69, v0.4.0 milestone), answering "does what my agents consumed match what the provider invoiced me?" outside the portal: GET /api/v1/usage/export?format=csv|jsonl streams every ledger row matching the #66 rollup's own filter set (plus afterSeq/asOfSeq for pinning and incremental pulls), and GET /api/v1/usage/export/totals returns the per-(currency, costSource) and per-token-kind reconciliation totals as the same GET /api/v1/usage/rollup aggregation reshaped, never summed a second time.

    • 28 fixed, versioned columns (schema agent-viewer.usage-export/1, in the X-Agent-Viewer-Export-Schema header and every JSONL record). Unknown stays unknown (empty CSV cell / JSON null, never 0); cost is written in plain decimal notation, never exponent form (1e-7 as 0.0000001), with parseFloat round-tripping exactly.
    • Every text column goes through the #68 redaction module individually (including each tag before joining), the only allowed source of text cells in the export; a redacted column says whether anything changed. CSV adds an OWASP formula-injection guard (a leading ' on a cell starting with = + - @, tab, CR, LF or a fullwidth form) and %/; tag escaping, both applied after redaction; JSONL carries the exact array and gets neither (never open a .jsonl export directly in a spreadsheet).
    • asOfSeq pins the response to one ledger snapshot (clamped to the current max, echoed back); afterSeq is the lower bound for "everything since my last pull". AGENT_VIEWER_EXPORT_MAX_ROWS (default 1,000,000) answers 422 export_too_large with the real row count before streaming anything, never truncating silently; AGENT_VIEWER_EXPORT_CONCURRENCY (default 2, the totals sidecar is exempt) answers 429 export_busy with Retry-After: 5. Neither route accepts token/api_key in the query (issue #71's shared guard already rejects it ahead of these routes); with no token configured, the same issue #71 loopback/Host/Origin guard already applies here too.
    • Streams without buffering the match set: SQLite mode pages through the ledger with keyset pagination on seq (reusing the #66 buildWhere filter builder), memory mode walks the ledger it already holds. A disconnected client stops the server from reading further; a mid-stream failure writes a JSONL export_aborted record (never a summary line) and destroys the socket rather than sending a silently-truncated file.
    • New usage_ledger.summary column (migration usage-ledger-summary): the export's own schema requires a redacted summary column sourced only from the ledger, and #65, as merged, never stored one. Nullable and never backfilled (the table is append-only): a row written before this migration keeps summary = NULL forever, like every other "legacy row has an empty cell for a field it never had" case this release already documents.
    • Fixes, discovered while building this: server/usage/rollup.ts's buildWhere/computeMemoryRollup gain an afterSeq lower bound (reused by the totals sidecar, not exposed on the rollup route itself); SQLiteEventStore's live-path ledger INSERT had silently drifted out of sync with usageLedger.ts's own column list.
    • Reference: docs/usage-export.md.
    • Deferred to a follow-up, out of this change's scope: the portal's own export dialog (UsageExportDialog.tsx) and its en/es strings; the 1,000,000-row nightly streaming variant (the 100,000-row PR-level budget is covered here); extending the golden reconciliation suite (#62) with export fixtures.

  • Usage ledger with server receive time (#65, first item of the v0.4.0 milestone): a new usage_ledger table holds one append-only, typed row per accepted llm.usage/llm.failed event, written in the same SQLite transaction as the event, carrying received_at (the server clock, always equal to that event's events.created_at) and occurred_at (the client's own timestamp). Every number is copied, never computed: an unreported token kind, cost, currency or latency is NULL, a reported 0 stays 0. A usage_ledger_skips table records why an accepted usage event produced no row instead: duplicate (same (provider, requestId), same figures), conflict (same key, different figures, with kept_event_id) or unparseable (only reachable from the backfill). Neither table has a foreign key to events, so retention (#70) will be able to prune old events without deleting their ledger rows. UPDATE usage_ledger always fails (usage_ledger is append-only).

    • GET /api/v1/events now returns receivedAt on every event (both storage modes); POST /api/v1/events and the batch route echo receivedAt too, with a duplicate reporting the original acceptance time, never the retry's. A client-sent receivedAt is always ignored on ingest.
    • New GET /api/v1/usage/ledger/status: row counts by origin (live/backfill), legacy rows, skip counts and the oldest/newest receive time. Never a sum of tokens or cost (that is #66).
    • Upgrade note: the first start of a 0.4.0 server runs a one-time backfill over every llm.usage/llm.failed event already stored, paged by rowid so memory stays flat on a large database, and logs one progress line per page plus a final summary, for example usage_ledger: scanned=1546 inserted=1530 duplicate=14 conflict=2 unparseable=0 (backfill). A backfilled row from a pre-0.4.0 server is marked legacy_contract = 1, which also resolves the 0.2.x "unknown reported as 0" ambiguity for cached/reasoning tokens (and, for legacy webhook rows only, input/output tokens) by storing those historical zeros as NULL. Every later restart repeats the same scan to catch up on any gap (for example after a temporary downgrade); in normal operation this finds nothing to do.
    • Memory mode gets its own independent ledger structures, capped by the new AGENT_VIEWER_USAGE_LEDGER_MAX_ROWS (default 100,000), never tied to AGENT_VIEWER_MAX_EVENTS's much smaller event ring: evicting an event from the ring never removes its ledger row.
    • Reference: docs/usage-ledger.md.

  • GET /api/v1/usage/rollup (#66, v0.4.0 milestone), the auditable read API over the usage ledger from #65: grouped sums for a question such as "how much did agent X spend with model Y this week", built to reconcile with a provider invoice.

    • groupBy takes 1 to 3 of agent, model, provider, session, task, day, user, tag; from/to (epoch ms, YYYY-MM-DD, or an offset-qualified ISO 8601) filter [from, to) on either received_at (server clock, the default) or occurred_at (timeBasis); utcOffsetMinutes is a fixed offset (no DST) for day buckets; every other filter (agentId, model, provider, sessionId, taskId, runtimeId, userId, tag, status, costSource, currency) is exact-match and repeatable (OR), never comma-split; asOfSeq pins the response to a past ledger position; sort is key (UTF-8 byte order, null last) or calls (descending); limit is 1 to 10,000. An unknown parameter, dimension or enum value, more than 3 dimensions, from >= to, an offset-less date-time, or more than 100 values in one filter all return 400 invalid_filter with a per-field issues array.
    • Costs are summed per (currency, costSource) pair and never merged across either axis; there is no currency conversion and no single "total cost". A null cost counts in unknownCostCalls instead of a sum. Each of the five token kinds (input, output, cacheRead, cacheWrite, reasoning) reports sum: null (never 0) when nobody reported it, with reportedCalls/unreportedCalls always adding up to calls.total. llm.failed rows count in calls.failed. A tag grouping is explicitly non-additive (groupsAreAdditive: false): a row with 2 tags lands in 2 groups, but totals still counts it once, over distinct rows with no tag join. truncated and groupCount say when limit cut off groups that totals still covers in full.
    • coverage never lets a partial answer look whole: complete turns false once the memory-mode ledger cap has refused rows that could belong to the query (droppedRows is the upper-bound count), or once retention (#70) has purged usage ledger rows the query range could reach (purgedThrough, the highest received_at purged so far, null until a retention run actually reaches a row); backfilledRows/legacyContractRows count matching rows that came from a pre-0.4.0 database.
    • New migration usage-rollup (schema version 8, since #67's own usage-calls-indexes migration already claimed version 7) adds usage_ledger_tags (one (ledger_seq, tag) row per tag, append-only, cascade-deleted with its ledger row) plus indexes on model, task_id and user_id, so every listed filter and both time bases resolve with a named-index SEARCH, never a bare table scan. Verified on a 100,000-row fixture: every single-dimension rollup over the full range finishes in well under the 2-second assertion budget (around 200-300ms in CI).
    • Memory and SQLite stores share one result shaper (server/usage/rollup.ts), so rounding (costs to 9 decimals), key ordering and day-bucket math cannot diverge between them. Shares parseUsageFilters (server/usage/filters.ts) with #67 (the per-call list below), extended with a userId/tag/asOfSeq/utcOffsetMinutes-only mode, so an identical filter query string selects the identical rows in both endpoints.
    • TypeScript SDK: AgentViewer.usageRollup(params), plus a host-only toUsageFigures(group, { costSource }) helper that maps one rollup group into the embeddable library's UsageFigures shape, returning cost: null whenever more than one currency, a cost-source mismatch, or any unknown cost would make a single number misleading. Python SDK: AgentViewer.usage_rollup(group_by, **filters) with snake_case filter kwargs; an unrecognized kwarg raises TypeError instead of being sent silently.
    • No change to @warlockcode/agent-viewer: no new props, no new aggregation, showUsage still defaults to off.
    • Reference: docs/integration.md.

  • GET /api/v1/usage/calls (#67): a read-only, metadata-only listing of the usage ledger from #65, with a stable, opaque cursor that can walk the full history in either direction (unlike GET /api/v1/events?type=llm.usage's afterId, which only moves toward newer events and returns whole events, including free text). Filters (agentId, sessionId, runtimeId, taskId, provider, model, status, costSource, currency, from/to/timeBasis) are parsed by the new, shared parseUsageFilters (server/usage/filters.ts), which the future rollup endpoint (#66) reuses so an identical query string selects an identical row set on both. Calls-only parameters: requestId, traceId, order (default desc), limit (1..1000, default 100) and cursor. Never returns summary, payload, prompt, completion, message, tool input/output or a provider error message: the serializer (toCallRecord, server/usage/calls.ts) is an explicit allow-list over an already-typed ledger row. A row created by the #65 backfill carries backfilled: true, returned exactly as stored, with no reinterpretation. No totals, sums or counts: that stays with #66. New SQLite migration 7 (usage-calls-indexes) adds the (agent_id, seq), (session_id, seq), (trace_id, seq) and (request_id) indexes the keyset query needs, plus a usage_ledger_meta table holding a persistent "store epoch" so a cursor keeps working across a SQLite restart (a memory-mode cursor from a previous process always answers 410 cursor_expired, since seq restarts there). Python list_calls()/iter_calls() and TypeScript listCalls()/iterateCalls() on both SDKs, the latter following nextCursor automatically and raising if the server ever returns the same cursor twice in a row. Reference: docs/integration.md.

  • Retention baseline: scheduled purge of events and the usage ledger, with GET /api/v1/admin/retention (#70): two independent windows, AGENT_VIEWER_RETENTION_DAYS (raw events) and AGENT_VIEWER_USAGE_RETENTION_DAYS (the usage ledger, defaults to keep forever), both measured by the server receive time, never the client's timestamp. Unset means keep; 0 and any other invalid value refuse to start the server, naming the variable and the accepted range. A scheduled job (server/retention.ts) runs once about 30 seconds after startup and then every AGENT_VIEWER_RETENTION_INTERVAL_MINUTES (default 60); with both windows unset it creates no timer at all. Deletes run in batches of 5000, each its own transaction, yielding to the event loop between batches so a large purge never blocks the SSE heartbeat; an overlapping tick is skipped, a failed run is logged and recorded without crashing the server, and the next tick retries.

    • Neither table has a foreign key to the other (new migration 8 also adds idx_events_created_at, retention_runs and retention_state, after #67's migration 7): purging events never changes a ledger row, a ledger-based rollup or any running total, and a re-sent event id whose row was purged is accepted again at the storage level but never adds a second ledger row or double-counts a total (toLedgerRow's id-based duplicate check and applyEvent's new skipUsage option, server/serverState.ts).
    • retention_runs keeps a self-pruned audit log of the latest 500 purge attempts (trigger, status, per-table cutoff and deleted count, a short content-free error on failure); retention_state persists each table's lifetime deletedTotal and purgedBefore (the largest cutoff under which rows were ever actually deleted, set before the first delete batch of a run so a crash never under-reports it) past that pruning and across a restart. A run left running by a dead process is closed as error/interrupted on the next open.
    • New GET /api/v1/admin/retention (same auth and rate limit as every other /api/v1 route, read-only, limit 1-200): counts, coverage signals and recent run history per table, with null (never 0) for an unconfigured window, an empty table's oldestReceivedAt, or a figure from a run that has not finished yet.
    • MemoryEventStore and SQLiteEventStore both implement the same purge()/retentionStatus() contract; the memory store's time-based purge is independent of its existing count-based AGENT_VIEWER_MAX_EVENTS eviction and never forgets a dedup id, exactly like that eviction already didn't.
    • docker/compose.yml forwards the three variables to the api service. Reference: docs/retention.md.

  • POST /api/v1/stream-tickets (#71): issues a short-lived (AGENT_VIEWER_STREAM_TICKET_TTL_MS, default 30 s, clamped to [1000, 300000]), single-use ticket for EventSource clients, capped at AGENT_VIEWER_STREAM_TICKET_MAX (default 1000) outstanding tickets at once (429 stream_ticket_limit past the cap). Requires the Bearer token; a ticket cannot mint tickets, and it only authenticates GET /api/v1/events/stream. A ticket minted under a token that is later rotated stops working, giving the operator a revocation path.

  • server/network.ts: isLoopbackHost, isLoopbackAddress, assertSafeBind and OpenApiRefusedError, the single definition of "loopback" the server's bind guard and open-mode request guard both use.

  • server/stream-tickets.ts: createStreamTicketStore, the in-memory, single-use ticket store behind POST /api/v1/stream-tickets.

  • Secret redaction module (issue #68, first slice of the 0.4.0 secret redaction baseline): a new, pure, dependency-free src/integrations/redaction.ts with no runtime imports (type imports only), so the server, the portal and the library will all be able to import the exact same rule set once they are wired to it.

    • BUILTIN_REDACTION_RULES (builtin-1): provider API keys (Anthropic, OpenAI, Google, Hugging Face, Groq, Slack, Stripe, GitLab, npm, GitHub), AWS access keys and secret keys, Azure storage account keys, Bearer and Basic auth headers, JWTs, PEM private key blocks, connection string passwords and generic key: value / key=value secret assignments. Every rule tolerates a 60- or 140-character truncation as long as the minimum body survives, and a masked placeholder such as ******** is never treated as a secret.
    • compileRedactionPolicy() builds a policy from the built-ins plus optional operator configuration: extra patterns (validated name, flags, capture group and a 50ms-per-pattern / 250ms-total startup self-test against a bundled adversarial corpus), disable for documented false positives, and ownSecrets for the server's own tokens (labelled own_secret, minimum 8 characters). Every active rule is also checked for idempotence on its own marker at compile time. Any violation throws RedactionConfigError naming the offending pattern, so a misconfigured server fails closed instead of starting with a partial policy.
    • redactText() replaces matches with a deterministic [REDACTED:<rule>] marker (keeping, for example, Bearer or user:@host) and is idempotent: redacting already-redacted text always reports count: 0.
    • truncateKeepingMarkers() cuts a string to a maximum length without ever splitting a marker in half.
    • redactEvent(), redactDeep() and findSecretsInIdentifiers() round out the module: event-level redaction that never touches llm.usage numeric fields or identifier keys (provider, model, requestId, currency, costSource, and the rest of the guarded list), an iterative deep-object string walker for arbitrary payloads, and an identifier guard that reports offending paths and rule names without ever returning the matched value.
    • New tests/redaction.test.mjs (94 cases, node:test): a true-positive and false-positive corpus per rule (built with runtime string concatenation, never literal secrets, so the repository stays clean for GitHub push protection and future gitleaks scanning), truncation, idempotency, determinism, the identifier guard, own_secret, extra-pattern validation (including a deliberately slow pattern that fails the startup budget), disable, the policy id hash, and loose-bound performance checks for a ~2KB event and a 1MB batch body.
    • Deferred to follow-up PRs under the same issue, per the issue's own suggested PR order (module, then server/store, then SDK/CLI/docs, then portal/library): wiring this module into the server's ingestion path (server/redaction.ts, the ingestEvents choke point, the 422 secret_in_identifier rejection, GET /api/v1/redaction/policy, /ready), the event_json/redaction_count SQLite migration and store API changes, the CanonicalEvent.redaction contract field, the three AGENT_VIEWER_REDACTION* environment variables and CLI flags, the SDK redactions pass-through, and the portal "mask secrets" wiring (useDisplayText, the shield badge, the library maskSecrets prop). Landing the engine first, fully tested and reviewed on its own, keeps this change small enough to review and avoids touching server/index.ts while issue #71 is also in flight on that file.

  • OTLP metrics ingestion (POST /v1/metrics) and http/protobuf for both OTLP routes (#73), building on the OTLP/HTTP logs receiver from #59. /v1/metrics stores Claude Code's claude_code.token.usage and claude_code.cost.usage counters as their own evidence (new telemetry_metric_points/telemetry_meta tables, migration 5), kept strictly apart from the ledger, the snapshot, SSE and every rollup: a metric point never becomes a canonical event. A keyed HMAC-SHA256 series key (random secret, persisted in SQLite mode) deduplicates retries and tells series apart without ever being recoverable from a stored attribute. Delta and cumulative aggregation temporality are both resolved correctly, including counter resets, through server/telemetry.ts (sumSeriesValue, sumTelemetryBySession), covered by a property test against random sequences.

    • Both /v1/logs and /v1/metrics now accept Content-Type: application/x-protobuf (OTLP's default wire format for most real exporters), decoded by a new, small, zero-dependency reader (server/otlp/protobufWire.ts, server/otlp/otlpProtobuf.ts) built from the opentelemetry-proto v1 field-number table: no protobuf library was added, so the runtime dependency set stays at express, lucide-react and zod. A protobuf request gets a protobuf response, including partialSuccess and error bodies. gRPC is not supported.
    • AGENT_VIEWER_TELEMETRY_MAX_POINTS (default 100000) bounds memory-mode storage of metric points; once full, further points are rejected through partialSuccess, never dropped silently. SQLite mode is bounded by disk, like the events table.
    • Deferred: GET /api/v1/usage/reconciliation (comparing the ledger's totals against these metrics) needs the usage ledger accessor from issue #65, which was still open when this shipped. sumTelemetryBySession is the metrics-side half that endpoint will consume once #65 lands.
    • Reference: docs/otlp.md.

  • Usage correlation and attribution fields (#64): llm.usage and llm.failed accept six new optional payload fields through a shared UsageCorrelationSchema block: traceId, parentId, toolCallId, meetingId, userId and tags. All are opaque to Agent Viewer (no aggregation, no UI, never summed or grouped by the embeddable library), additive and optional, so every event that validated before still validates with the same output.

    • Each id field accepts 1 to 128 characters, no control characters and no leading or trailing whitespace (the exact ECMAScript trim() whitespace set); tags allows up to 20 items of 1 to 64 characters each, deduplicated in order after the 20-item limit is checked. null and absent both mean "not reported" and are omitted from the stored payload; an empty string is rejected, not silently accepted.
    • Reused automatically by the generic webhook's usage object (it already shares LlmUsagePayloadSchema), parseEventLog (JSONL import), the SSE stream and both the memory and SQLite stores: no storage migration needed.
    • The Python SDK's usage(), the new llm_failed() helper and the legacy llm_usage() accept the six fields as keyword-only arguments (snake_case), validate them locally with the same limits (ValueError naming the argument, TypeError for a wrong type), and count length in UTF-16 code units to match the server exactly. tool_started/tool_completed/tool_failed accept tool_call_id.
    • The TypeScript SDK's usage(), the new llmFailed() helper and the three tool helpers accept the same fields, validated locally with AgentViewerError (issues in the same { path, message } shape the server returns).
    • The five example adapters (openai-agents-adapter.ts, langgraph-adapter.ts, google-adk-adapter.ts, autogen-adapter.py, crewai-adapter.py) forward the trace, parent and tool call ids their framework exposes, never inventing one, and never forward framework tags or metadata automatically.
    • CORRELATION_ID_MAX_LENGTH, USAGE_TAGS_MAX, USAGE_TAG_MAX_LENGTH and the UsageCorrelation type are exported from @warlockcode/agent-viewer. A 0.4.0 or later server is needed to keep these fields: an older server accepts the event and silently drops them.
    • See docs/integration.md for the full contract.

  • The live portal loads its history before subscribing to the stream (#72), so a reload or a second tab opened mid-run restores the office and the token/cost figures from the server instead of starting at zero. On open, the portal fetches GET /api/v1/snapshot (reusing the rebuild path issue #54 already added for a resync) and, when the store holds more than the snapshot's own newest 100 events, pages further back with the new beforeId parameter of GET /api/v1/events up to VITE_AGENT_VIEWER_HISTORY_LIMIT (default 1000, valid range 1 to 5000) so the activity timeline can go deeper; figures never depend on that extra depth, only the snapshot's own totals. Only after that does it subscribe to GET /api/v1/events/stream, resuming from the snapshot's own lastEventId through the library's existing lastEventId option. The top bar badge now reflects this as a loading / subscribing / live / reconnecting / error phase (data-testid="live-phase", data-phase), with a retry control when the history fails to load; it previously only reacted to the arrival of an event.

  • GET /api/v1/events now supports backward paging: a beforeId cursor (events stored strictly before it, in arrival order), hasMore and nextBeforeId in the response, so a client can walk the full stored history with no duplicates and no gaps on both the memory and the SQLite store.

  • parseEventLog reads an OTLP/JSON file (#74), not only canonical JSONL: a pretty-printed OTLP export, or one request per line (an OpenTelemetry Collector file exporter). resourceLogs records convert to llm.usage/llm.failed events through the same Claude Code mapper the live POST /v1/logs receiver uses (now shared at src/integrations/otlp/claudeCodeLogs.ts, moved out of server/ so the library can bundle it); a record with no usable timestamp or a required field missing is rejected and counted, never given the current time or a made-up figure. resourceMetrics is never replayed as calls (metrics are pre-aggregated counters): one clear issue per file explains why, pointing at the logs signal and at POST /v1/metrics. resourceSpans support is unchanged (not yet available). EventLogParseIssue gains optional code (EventLogIssueCode: invalid-json, invalid-event, otlp-metrics-not-replayable, otlp-traces-not-supported, otlp-record-skipped, otlp-no-usage-records) and path; EventLogParseResult gains an optional otlp summary (signals, logRecords, converted, skipped, rejected). An OTLP issue never sets raw, since a log record can carry sensitive text. The demo app now picks a localized drop.* message from the issue codes instead of always showing the generic "no events" text. Reference: docs/event-log.md.

    • src/integrations/claudeCodeIdentity.ts now hashes with a new pure-JS sha256Hex (src/integrations/sha256.ts) instead of node:crypto, so the shared OTLP mapper has no node:* import and stays bundleable for the browser; the digest is byte-identical to the previous node:crypto-based one, so every existing id (CLI hook, live OTLP receivers) is unchanged.

Fixed

  • Release pipeline hardening, a direct follow-up to the 0.3.0 release gate (#63): .github/workflows/release.yml no longer lets a tag "succeed" with nothing published.
    • The npm publish step now fails the job (instead of a silent exit 0) when NPM_TOKEN (or trusted publishing) is missing on a real release tag.
    • A new pypi job builds the sdist and wheel from sdk/python/ with python -m build, checks them with twine check, installs the wheel in a clean venv and publishes to PyPI with pypa/gh-action-pypi-publish (trusted publishing, pypi environment).
    • A tag with an -rc.N, -beta.N or -alpha.N suffix (for example v0.3.0-rc.1) is now detected as a prerelease: every job still builds and checks everything, but nothing publishes to npm, PyPI or GHCR, and the GitHub release is created as a prerelease. The detection lives in scripts/release-rules.mjs and is exposed as the release job's is_prerelease output so every other job reads the same value.
    • The GHCR docker job now pushes only on a non-prerelease tag (it still builds both images on a prerelease, to verify they build) and only after npm and PyPI have finished.
    • The GitHub release notes are now the real ## [X.Y.Z] section extracted from CHANGELOG.md via --notes-file, instead of the fixed text "See CHANGELOG.md".
    • The version-alignment check now also fails when sdk/python/pyproject.toml disagrees with the tag, not only package.json.
    • New tests/release-versions.test.mjs and tests/release-workflow.test.mjs cover this logic (node:test), importing the shared scripts/release-rules.mjs rather than re-implementing it, and asserting on the workflow's actual job structure and shell scripts.

Changed

  • GET /api/v1/events's limit is now clamped to [1, 500] (Math.floor, non-numeric or absent still gives the previous default of 100); it was previously unbounded and unchecked.
  • The SQLite event store now orders GET /api/v1/events by its durable arrival sequence (seq) alone, the same order the memory store and the SSE reconnect replay already use, instead of timestamp first. This only matters when events arrive with a non-monotonic timestamp (a backfilled batch, more than one runtime, a clock correction): before this change a beforeId/afterId cursor could disagree with the listed order in that case and a backward walk could duplicate or skip a row.
  • parseEventLog classifies its input by parsing, not by a substring test (#74): a canonical JSONL line whose summary happens to contain the text "resourceSpans" now parses as JSONL instead of being misrouted to the OTLP branch. OTLP input now reports the real physical line count in totalLines (previously hard-coded to 1), and a truncated pretty-printed OTLP document (for example cut off mid-export) now gives one invalid-json issue instead of one "Malformed JSON" issue per physical line.
  • The release pipeline no longer treats npm and PyPI as required. The GitHub release (with the packed .tgz) and the GHCR Docker images are the actual release; npm and PyPI publishing are optional extras. The "Publish to npm" and "Publish to PyPI" steps are now continue-on-error: true: a missing NPM_TOKEN or unconfigured PyPI trusted publisher still prints a clear error annotation and shows as a failed step in the run, but no longer fails the job or blocks the GitHub release / Docker images from being created. Publishing to npm and PyPI for real is tracked separately as a future enhancement, not a release blocker.
Assets 5
Loading