Refactor around returning and using AgentTrace
#202
Conversation
Move `any_agent/tracing.py` to `any_agent/tracing/tracer.py`.
…etry-agentresult-and-anyagenttrace
There was a problem hiding this comment.
Pull Request Overview
This PR refactors the tracing and evaluation logic to consolidate telemetry, agent result, and agent trace functionality. Key changes include removing the legacy telemetry modules, updating agent framework methods to return AgentTrace instead of AgentResult, and refactoring tracing and instrumentation initialization across the codebase.
Reviewed Changes
Copilot reviewed 34 out of 34 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/any_agent/tracing/exporter.py | Introduces the new AnyAgentExporter for building and exporting traces. |
| src/any_agent/tracing/init.py | Updates the public API of the tracing module. |
| src/any_agent/tracing.py | Removes the legacy tracing module now replaced by the new implementation. |
| src/any_agent/telemetry/* | Removes telemetry cost and processor files as they are consolidated. |
| src/any_agent/frameworks/*.py | Updates run_async methods and imports across frameworks to use AgentTrace. |
| src/any_agent/frameworks/any_agent.py | Refactors the AnyAgent base class to use the new _setup_tracing and exporter model. |
| src/any_agent/evaluation/* | Adjusts evaluator and evaluation modules to use the new tracing processor. |
| src/any_agent/config.py | Improves the validation error message for output_dir when saving traces. |
| src/any_agent/init.py | Updates public imports to expose AgentTrace and remove legacy types. |
| README.md | Updates usage examples to reflect the new tracing interface. |
Comments suppressed due to low confidence (1)
src/any_agent/frameworks/tinyagent.py:191
- After removing the call to self._create_tracer, please verify that self._exporter is reliably initialized via _setup_tracing before it is used in run_async.
async def run_async(self, prompt: str, **kwargs: Any) -> AgentTrace:
| from opentelemetry.sdk.trace import ReadableSpan | ||
|
|
||
|
|
||
| class AnyAgentExporter(SpanExporter): |
There was a problem hiding this comment.
@njbrake You could implement here the:
include things like the agent framework that was used, which would be a much more reliable method to load the trace when it comes to evaluation
However, let's make sure that the resulting JSON is compatible with OTEL, whatever that means
daavoo
changed the title
AgentTrace
daavoo
deleted the
194-refactor-to-consolidate-evaluation-telemetry-agentresult-and-anyagenttrace
branch
| self._last_tracer: Tracer | None = ( | ||
| None # holds the trace from the most recent run | ||
| ) | ||
| self._tracing_config = tracing |
There was a problem hiding this comment.
@daavoo Whoops I missed this during the initial review. Making this change means that now all the agent.run() commands from an agent are going to get merged into a single trace. That's going to be an issue for people that are running the agent in a server or want to have multiple runs happening in a single instantation of the agent.
result = agent.run("What's the best agent framework")
result2 = agent.run("What's the best agent framework")
result2 is going to be result and result2 appended on top of it
TODO:
tracing/processors