Skip to content

Consider pruning failed structured output and tool call generations #495

@eb8680

Description

@eb8680

When Templates generate malformed tool calls or structured outputs detectable statically at the level of the tool signature or output schema, we should probably be removing the failures early from the message sequence once a corrected version is generated in order to minimize context usage.

Note this is distinct from semantic failures in tool calling, where we may generate a correctly typed tool call that fails when executing the body of the tool. These need to persist for the duration of a Template call.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions