Record instructions on the agent run span even when they are dynamic #3131
Conversation
Docs Preview
|
| def _run_span_end_attributes( | ||
| self, state: _agent_graph.GraphAgentState, usage: _usage.RunUsage, settings: InstrumentationSettings | ||
| self, | ||
| state: _agent_graph.GraphAgentState, |
There was a problem hiding this comment.
Since we only use state.message_history, lets send in just that, so that the compatibility between that list and new_message_index are clear.
|
|
||
| # Store an attribute that indicates that the instructions from this agent run were not always the same | ||
| # This can signal to an observability UI that different steps in the agent run had different instructions | ||
| # Note: We purposely only look at "new" messages because they are the only ones produced by this agent run. |
There was a problem hiding this comment.
Should we also store the new_message_index so that the UI can show which messages are from older agent runs? Similar to how I think we should show the ModelResponse.model_name if it's not the same between different messages
| for m in state.message_history[new_message_index:]: | ||
| if ( | ||
| isinstance(m, _messages.ModelRequest) | ||
| and m.instructions is not None | ||
| and m.instructions != last_instructions | ||
| ): | ||
| attrs['pydantic_ai.variable_instructions'] = True |
There was a problem hiding this comment.
Can we simplify this into an any()?
There was a problem hiding this comment.
I don't understand this suggestion
There was a problem hiding this comment.
@dmontagu attrs['pydantic_ai.variable_instructions'] = any(isinstance(m, _messages.ModelRequest) and m.instructions is not None and m.instructions != last_instructions for m in state.message_history[new_message_index:])
dmontagu
force-pushed
the
dmontagu/record-instructions-on-agent-run-span
branch
from
bbd4409 to
84361ab
Compare
2 participants
This does the following:
pydantic_ai.variable_instructionstoTrueas a way to have a signal in a UI like Logfire that the agent run made use of different instructions in different steps.Closes #2884