Skip to content

usage estimation: wire the non-streaming ensemble sub-call events (panel + judge) #796

Description

@jarvis9443

Follow-up to #794 (AISIX-Cloud#1074). The non-streaming ensemble path emits one usage event per panel member plus the judge (emit_subcalls in chat.rs), but none of those events go through the token-estimation fallback: with a no-usage backend they still record silent zeros.

The streaming ensemble judge IS wired (#794 arms the SSE Drop guard with the judge's own request). Wiring the non-streaming side needs the judge request and the per-member answer texts at the emit site, which ensemble.rs currently keeps internal — that refactor was out of scope for the #794 review fixes (audit finding L2, deferred explicitly).

Scope:

  • panel member events: prompt from the client request, completion from the member's buffered answer text
  • judge event: prompt from judge_request(...) output, completion from outcome.response text
  • both flagged usage_estimated per the feat(usage): estimate tokens locally when the upstream reports no usage #794 or-semantics helper
  • test: wiremock ensemble with a usage-less backend asserting non-zero flagged sub-call events

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions