Skip to content

0.69.16

Choose a tag to compare

@pchalasani pchalasani released this 25 Oct 20:01
· 4 commits to main since this release

Content-filter stops are now reported as metadata, not as model text

LLM response handling

Since 0.69.0, when a provider's content filter (e.g. Azure OpenAI) stopped a
streamed reply, Langroid synthesized the text
Cannot respond due to content filters [...] and streamed it as if the model
had said it. That notice was spoken aloud by TTS in voice apps, stored in
message_history, and fed back to the model on later turns — and the handling
only covered the streaming path at all.

A content-filter stop is now surfaced as structured metadata instead
(#1277, fixes #1276):

  • LLMResponse.finish_reason and ChatDocument.metadata.finish_reason are
    populated for streaming and non-streaming, sync and async, and are preserved
    on cache hits.
  • The stream processors no longer inject synthesized text; the filter
    categories go to the log.
  • The re-send loop from #658 stays fixed one level up: the new
    content_filter_message config replaces an empty filtered reply in the
    returned ChatDocument only — it is never streamed and never enters history.
    Setting it to None leaves the reply empty for applications that want to
    handle the stop themselves.
  • content_filter_message lives on AgentConfig with the fallback in Agent,
    so a Task running on a plain Agent behaves correctly too. ChatAgent
    inherits both; no change is needed for existing ChatAgent code.

Covered by 37 new tests in test_chat_agent_content_filter.py and
test_openai_gpt_none_content.py.

Migration note: applications that matched on the literal
Cannot respond due to content filters string should switch to checking
finish_reason == "content_filter" (or set content_filter_message to the
text they want).

Thanks to @alexagr for reporting #1276 and for the original implementation,
whose commit is carried in this release with authorship intact.

Full changelog: 0.69.15...0.69.16