Tracing LLM executions

[BardoUntiveros]

Is there a way to trace LLM execution details in the workflow web UI?
If a step is generating a text, I can only see input and output, but no more details, like tool calls.

[pranaygp]

I'd recommend using the AI package (https://useworkflow.dev/docs/api-reference/workflow-ai). This will automatically split every tool call and llm fetch into its own step. Example usage here

Related X Post: https://x.com/pranaygp/status/1983647248463638891

This way, every step input/output will get captured in the o11y UI, and since every tool call is a step, you should have all the o11y there.

There will be more work still to make the DX even better and help you automatically trace tokens spent/etc. (likely using the tags spec)