Severity: Medium
Model: sonnet
Severity: Medium
Where: tokenjam/ — tokenjam/core/ingest.py (_is_session_end) + alert engine session-end gating
Observed: SESSION_DURATION and COST_BUDGET alerts only evaluate on session end, and session end is only detected via an invoke_agent span with real duration. Any SDK/OTLP ingest path that emits only gen_ai.llm.call spans (e.g. generic OTLP exporters, the HAL benchmark replayer, likely many raw-SDK integrations) can never trigger these alert types: sessions never 'end', so the checks are structurally unreachable. Confirmed during benchmark replay: 21,562 spans, 0 session-end alerts possible by construction.
Fix direction: Add an inactivity-based session-end fallback (e.g. N minutes with no new spans closes the session and runs end-of-session checks), or evaluate duration/cost-budget checks incrementally on span ingest past thresholds. Keep invoke_agent as the fast path.
Done when: an llm.call-only span stream that exceeds a configured session duration/cost budget raises the corresponding alert without any invoke_agent span, with tests.
Ref: SDK benchmark dogfood (HAL trace replay), 2026-07-17.
Severity: Medium
Model: sonnet
Severity: Medium
Where: tokenjam/ — tokenjam/core/ingest.py (_is_session_end) + alert engine session-end gating
Observed: SESSION_DURATION and COST_BUDGET alerts only evaluate on session end, and session end is only detected via an invoke_agent span with real duration. Any SDK/OTLP ingest path that emits only gen_ai.llm.call spans (e.g. generic OTLP exporters, the HAL benchmark replayer, likely many raw-SDK integrations) can never trigger these alert types: sessions never 'end', so the checks are structurally unreachable. Confirmed during benchmark replay: 21,562 spans, 0 session-end alerts possible by construction.
Fix direction: Add an inactivity-based session-end fallback (e.g. N minutes with no new spans closes the session and runs end-of-session checks), or evaluate duration/cost-budget checks incrementally on span ingest past thresholds. Keep invoke_agent as the fast path.
Done when: an llm.call-only span stream that exceeds a configured session duration/cost budget raises the corresponding alert without any invoke_agent span, with tests.
Ref: SDK benchmark dogfood (HAL trace replay), 2026-07-17.