agentlog-llm

Trace AI agent execution — LLM calls, tool calls, costs and latency. Zero dependencies.

pip install agentlog-llm

The Problem

Agentic AI systems make multiple LLM calls, use tools, make decisions — all invisibly. When something goes wrong (or costs too much), you have no idea what happened inside.

agentlog-llm gives you a complete execution trace — every LLM call, every tool use, every decision — with costs, latency, and token counts.

Quick Start

from agentlog import AgentTracer

tracer = AgentTracer("my-agent", task="Summarize documents")

# Log an LLM call
with tracer.llm_call("openai", "gpt-4o", prompt="Summarize this...") as call:
    response = openai_client.chat(...)
    call.response = response.text
    call.tokens_in = response.usage.prompt_tokens
    call.tokens_out = response.usage.completion_tokens

# Log a tool call
with tracer.tool_call("web_search", inputs={"query": "AI news"}) as tool:
    result = search("AI news")
    tool.output = result

# Log a decision
tracer.log("Decided to summarize top 3 results only")

# Finish and get trace
trace = tracer.finish(success=True)
print(trace.summary())

Output:

{
    "trace_id": "a1b2c3d4",
    "agent": "my-agent",
    "task": "Summarize documents",
    "steps": 3,
    "llm_calls": 1,
    "tool_calls": 1,
    "total_tokens": 1250,
    "total_cost_usd": 0.00425,
    "total_latency_ms": 2340.5,
    "success": True
}

Decorator Support

@tracer.trace_tool("calculator")
def calculate(expression: str) -> float:
    return eval(expression)

# Tool call is automatically logged
result = calculate("2 + 2")

Export

# As dict
data = tracer.to_dict()

# As JSON string
json_str = tracer.to_json()

# Save to file
tracer.save("trace.json")

AgentTrace Properties

trace.total_llm_calls     # Number of LLM calls
trace.total_tool_calls    # Number of tool calls
trace.total_tokens        # Total tokens used
trace.total_cost_usd      # Total cost in USD
trace.total_latency_ms    # Total latency in ms
trace.llm_calls           # List of LLMCall objects
trace.tool_calls          # List of ToolCall objects
trace.summary()           # Dict summary

License

MIT