Observability for 9-agent teams: what metrics actually matter?

[avoronov-explyt]

At Aries AI we run 9-agent teams that build products autonomously. After months of iterating on our observability stack, here are the metrics that actually moved the needle:

Agent-level:

• Task completion rate per role (not just overall)
• Time-to-first-output (catches stuck agents early)
• Self-correction rate (agent fixes its own output before handoff)
• Token efficiency per completed task (cost per unit of value)

Team-level:

• Handoff latency (time between agent A finishing and agent B starting)
• Blocker propagation time (how fast does a blocker reach the manager agent?)
• Queue depth per agent (are we bottlenecking on one role?)
• Cross-session drift (do agents maintain consistent behavior across sessions?)

What did NOT matter:

• Total tokens consumed (vanity metric without per-task context)
• Raw error count (without severity and recovery classification)
• CPU/memory usage (agents are I/O bound, not compute bound)

We use AgentOps for session-level tracing and built a custom dashboard on top for the team-level metrics. Would love to see more of these land as first-class features. What metrics are you all tracking?