AgentForge 🔭

Datadog × LangSmith — but open-source, self-hosted, and built for the agent community.

AgentForge is an open-source AI Agent Observability & Production Runtime Platform. It gives you the tools to trace, monitor, and debug agent decision chains — without being locked into a proprietary vendor.

Why AgentForge?

Building AI agents is easy. Running them in production isn't. Every agent call — LLM inference, tool execution, decision branch — is a potential failure point. Without observability, you're debugging blind.

AgentForge gives you:

• Tracing SDK — lightweight OpenTelemetry-compatible wrappers for OpenAI, LangChain, and CrewAI
• Live Dashboard — real-time DAG visualization of agent decision chains
• Cost Tracking — token usage and cost per agent run
• Self-Hosted — deploy on your own infrastructure. No data leaves your control.

Quick Start

pip install agentforge

Trace an OpenAI Agent

from agentforge import trace
from openai import OpenAI

client = OpenAI()

# Wrap any OpenAI call
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}],
)

# That's it. AgentForge captures timing, tokens, and the full trace.

Trace a LangChain Agent

from agentforge.langchain import AgentForgeCallbackHandler
from langchain.agents import initialize_agent, Tool

handler = AgentForgeCallbackHandler()

agent = initialize_agent(
    tools=[Tool(name="search", func=lambda q: "results")],
    llm=...,
    agent="zero-shot-react-description",
    callbacks=[handler],
)

Run the Dashboard

# Start the tracing server (default: http://localhost:8000)
python -m agentforge.exporter serve

# In another terminal, run your agent with AGENTFORGE_ENDPOINT set
AGENTFORGE_ENDPOINT=http://localhost:8000/traces python my_agent.py

# Open http://localhost:8000 in your browser

Architecture

┌─────────────┐     ┌──────────────┐     ┌──────────────┐
│ Your Agent  │────▶│  AgentForge  │────▶│   Dashboard  │
│ (OpenAI /   │     │   Tracing    │     │  (FastAPI +  │
│  LangChain) │     │    SDK       │     │    React)    │
└─────────────┘     └──────────────┘     └──────────────┘
                           │
                           ▼
                    ┌──────────────┐
                    │  Console /   │
                    │  File / HTTP │
                    │  Exporters   │
                    └──────────────┘

Features

• ✅ OpenAI interceptor — monkey-patches ChatCompletion.create for zero-code instrumentation
• ✅ LangChain callback handler — plug into any LangChain agent/chain/tool
• ✅ OpenTelemetry spans — compatible with any OTLP-compatible backend
• ✅ DAG trace visualization — see the full decision tree in the dashboard
• ✅ Token cost tracking — per-run and aggregate cost analysis
• ✅ Console, file, and HTTP exporters — flexible output destinations
• ✅ Self-hosted dashboard — FastAPI backend + React frontend

Roadmap

• CrewAI integration
• Custom span attributes for agent reasoning
• Alerting and threshold-based monitoring
• Persistent storage (SQLite → PostgreSQL)
• Agent replay — replay traces against different models

🌐 Ecosystem

AgentForge is part of the Cubiczan open-source ecosystem:

Repo	Description
🧠 CHP	Decision-governance for multi-agent systems — adversarial review, lock progression
💼 MeshCFO	Multi-Agent CFO OS — board-ready financial analysis
📊 SEC Workbench	CHP-hardened SEC filing research platform

Agent observability + consensus hardening = enterprise-trustworthy AI agents.

🤝 Contributing

Check out good first issue labels to get started. PRs, issues, and ideas welcome.

License

MIT