Looking for collaborators: AI Agent Runtime Safety Monitoring & Diagnostic Framework

[wwreixi]

🏷️ Discussion Type

Question

Body

We're building an open-source runtime safety layer for AI Agents — think of it as a "health monitoring system" that detects when an Agent is going off-track, losing stability, or operating outside safe boundaries before it fails.

Project repo: https://github.com/wwreixi/agent-trace-diagnostics

What it does:

• Injects lightweight hooks into Agent execution (CrewAI currently supported)
• Computes real-time health scores (S) based on Transformative force (T), Constraint force (C), and Internal dissipation (D)
• Outputs structured diagnostic signals: S/T/C/D, alert zones (Green/Yellow/Orange/Red), and navigation instructions
• Standardized evaluation via DeepEval: 12 test cases, 100% pass rate

Current status:

• ✅ Framework design complete
• ✅ Langfuse integration for trace storage
• ✅ FastAPI endpoints (/diagnosis, /batch-diagnosis, /stats)
• ✅ 12 benchmark test cases covering normal, edge, and security scenarios
• 📍 Actively seeking real-world validation and co-development

Who we're looking for:

Type	What we need
Agent operators	Teams running production or staging Agents (any framework: CrewAI, LangChain, AutoGPT, etc.). Share anonymized traces to help us calibrate thresholds and validate the diagnostic model
Python engineers	Help build framework adapters, optimize the diagnostic engine, and improve the evaluation harness
Safety/formal methods researchers	Interest in behavior dynamics, runtime verification, or control theory — we're exploring rigorous foundations for Agent safety

How to get involved:

1. Star & watch the repo
2. Open an Issue with questions or suggestions
3. Reach out directly: [Personal info edited by staff]

We're open to all forms of collaboration — code contributions, testing partnerships, feedback, or just a conversation to explore alignment.

Guidelines

• I have read and understood this category's guidelines before making this post.

[AhmadHassan-BTed]

I'm currently running an agentic extractor that evaluates web content and live incoming chat messages against custom scoring matrices. This runtime safety layer is exactly what I need to catch when the agent goes off-track.

Since my core framework is mostly Python, I can easily help build out some of the framework adapters or just plug your hooks into my setup to feed you some real-world staging traces. I'd love to see how your S/T/C/D scoring holds up against my edge cases.

Starred the repo. I'll shoot you an email so we can see if it's a good fit.

[wwreixi]

Hi Ahmad, thanks for your response. I'm glad the runtime safety layer aligns with your use case. Let's move the conversation to email for more detail — you can contact me directly at [Personal info edited by staff]. Hope to hear from you soon!

[lgustavopuga]

This is incredibly timely! As AI agents gain autonomy, real-time safety guardrails are the biggest bottleneck keeping them out of production.

For anyone landing on this thread looking to collaborate, here is how you can plug in today:

• Operators (CrewAI, LangChain, etc.): Integrate the lightweight hooks to test the health score ($S$) against your edge cases. Sharing anonymized traces is the fastest way to calibrate the S/T/C/D thresholds.
• Python Engineers: Help expand framework support beyond CrewAI, optimize the FastAPI endpoints, or refine the DeepEval harness.
• Safety Researchers: Help ground the S/T/C/D metrics (Transformative, Constraint, and Dissipation forces) in control theory or formal runtime verification.

👉 How to join: Star the [agent-trace-diagnostics](https://github.com/wwreixi/agent-trace-diagnostics) repo, open an issue with your ideas, or reach out directly at [Personal info edited by staff]!

[v-coder247]

Hi @lgustavopuga ,

I've removed your private information from [this post](link to discussion) as posting identifiable information is not allowed, per our Code of Conduct.

[v-coder247]

Hi @wwreixi,

I've removed your private information from [this post](link to discussion) as posting identifiable information is not allowed, per our Code of Conduct.