Capture metrics from workflow running inside a tool

[raphaelrodrigues]

Hi everyone,

I’m running into an issue where AgentOS is not capturing the token metrics from a workflow that runs inside a custom tool.

My Setup
I have a Team with a custom async tool that internally runs a Workflow:

@tool
async def process_documents(
    run_input: RunInput,
    run_context: RunContext,
) -> AsyncGenerator[Any, Any]:
    
    workflow = my_workflow()
    
    async for event in workflow.arun(
        input=run_input.input_content,
        session_state=run_context.session_state,
        session_id=run_context.session_id,
        stream=True,
        stream_events=True,
    ):
        yield event


my_team = Team(
    name="Document Processing Team",
    tools=[process_documents],
    # ... other config
)

The Problem
When I print the metrics from the workflow events, I see the actual token usage:

{'input_tokens': 4201, 'output_tokens': 2288, 'total_tokens': 6489}
{'input_tokens': 4034, 'output_tokens': 2137, 'total_tokens': 6171}
{'input_tokens': 3720, 'output_tokens': 1142, 'total_tokens': 4862}

But in AgentOS, I only see:

Input Tokens: 4,960
Output Tokens: 145
Total Tokens: 5,105

AgentOS is only tracking the outer Team’s metrics and completely missing the ~17,500 tokens consumed by the inner workflow agents.

Questions

Is there a way to have nested workflow metrics automatically propagate to AgentOS?

Would using WorkflowTools instead of a custom tool solve this?

Is this a known limitation or am I missing something?

This is important for tracking actual token usage and costs. Any help appreciated!

Thanks!

[aniruddhaadak80]

From my point of view, the missing piece is hierarchical accounting rather than raw metric collection. If a workflow runs inside a tool, AgentOS should either treat it as a child trace of the outer run or offer an explicit rollup API so the parent team run can absorb nested token usage instead of silently underreporting it.

Without that, cost data becomes misleading exactly in the workflows where orchestration is most valuable.

[kausmeows]

This is a missing gap @raphaelrodrigues, thanks for the spot. We're working on how to best enable this. Will keep you posted!

[omni-front]

not sure about this one tbh, but it seems like using WorkflowTools might help since they could be more integrated with AgentOS. you might want to check if there's a way to explicitly report the inner workflow's metrics to AgentOS.

[raphaelrodrigues]

Hello @kausmeows ,
Do you have any planned date for when this enhancement will be addressed?
At the moment, AgentOS is missing the output‑token metrics, which makes the token tracking quite incomplete on our side. We are very happy with AgentOS overall, but this gap makes it difficult to present accurate statistics.
We also reviewed the WorkflowTools code, and it seems that switching to them would not resolve the issue.
It would be great to know if this is already on the roadmap or if there is an expected timeline for the fix.
Thanks!

[kausmeows]

hi @raphaelrodrigues we're planning this with a major release of 2.7.0 this month. There are several parts to it-

1. capture metrics from agent/team/workflow run inside a custom tool
2. for custom step function in workflow if an agent/team/workflow is run there, we should also capture that.

This is a relatively complex one and we're planning to pool this in with a major release!

[raphaelrodrigues]

Thanks so much for the update and for the amazing support as always!
Really excited to see this coming in 2.7.0!

[omni-front]

I'm not aware of any specific timeline for this enhancement, but I'll pass along your feedback to the team to see if it can be prioritized.

[jagmarques]

This is a known gap - nested workflow metrics do not propagate to the parent team run in AgentOS. The issue is that the inner workflow creates its own metric context that is not linked to the outer team's trace.

Until the official fix ships, you can work around it by manually aggregating the metrics. After the inner workflow completes, sum up the token counts from its events and attach them to the parent run context using run_context.session_state to pass the totals back up. Not elegant, but it gives you accurate cost tracking in the meantime.

[omni-front]

Ah, good catch, you're right. Thanks for sharing the workaround with manual aggregation!