💡 AI/LLM Observability Workflow Extension for Morgan

[github-actions[bot]]

Summary

Extend Morgan's observability workflow (CO) to cover AI/LLM-specific monitoring: OpenTelemetry GenAI semantic conventions, LLM golden signals (token throughput, latency-per-token, cost-per-query), hallucination rate SLIs, model drift detection, and AI agent trace visualization. As 40% of enterprise apps embed AI agents by 2026, observability planning that covers traditional services but ignores AI workloads leaves teams blind to their fastest-growing operational risk.

Market Signal

OpenTelemetry GenAI semantic conventions are now standardized with auto-instrumentation packages for OpenAI, Anthropic, LangChain, and LlamaIndex. Datadog Watchdog, New Relic, and every major observability platform now support AI-specific dashboards. PagerDuty and AWS DevOps Agent correlate LLM telemetry with incident data. The industry has moved from "should we monitor AI?" to "how do we monitor AI systematically?" — the planning methodology gap is where Morgan adds value.

User Signal

The project's observability workflow (CO) currently covers only traditional service monitoring (golden signals, RED/USE methods). As BMAD users increasingly build AI-powered applications (the BMAD ecosystem is 49K+ stars and growing rapidly), the lack of AI-specific observability guidance is a significant gap. The existing GenAI semantic conventions make this immediately actionable.

Technical Opportunity

The observability workflow already has a multi-step structure (step-01-init through step-05-validation) with templates. Adding AI observability sections to step-02 (metrics strategy) and step-03 (alerting strategy) is a natural content extension. Morgan's existing golden signals framework can be extended with AI-specific signals. The production readiness checklist template can add AI observability verification items.

Assessment

Dimension	Score	Rationale
Feasibility	high	Content-only extension to existing workflow steps and templates — no new architecture required
Impact	high	Addresses a growing gap for every BMAD user deploying AI/ML workloads; OpenTelemetry GenAI conventions make it immediately actionable
Urgency	high	GenAI semantic conventions are standardized NOW; 40% of enterprise apps embedding AI agents by 2026 (Gartner); competitive observability tools already supporting AI dashboards

Adversarial Review

Strongest objection: OpenTelemetry docs and vendor platforms already provide AI observability guidance. Teams can instrument without Morgan's workflow.

Rebuttal: Tool vendors provide instrumentation, not planning methodology. No tool tells teams which SLIs to define for their specific AI use case, how to set error budgets for hallucination rates, how to structure alerting for model drift vs. data drift, or how to design dashboards for LLM cost visibility. That architectural planning gap — deciding WHAT to observe before HOW to instrument — is precisely Morgan's value proposition, consistent with how the existing workflow handles traditional observability.

Suggested Next Step

Add conditional AI/LLM observability sections to the existing bgr-3-create-observability workflow steps (step-02 metrics strategy, step-03 alerting) and extend the observability plan template with an AI/LLM monitoring section. Gate activation on a new config flag in module.yaml (e.g., ai_workloads: true/false).

[github-actions[bot]]

Weekly Update

What Changed

• OpenTelemetry graduated to CNCF Graduated status in May 2026 — solidifying it as the de facto observability standard. This significantly strengthens the case for building Morgan's AI observability extension on OTel GenAI semantic conventions, as the standard is now production-stable with long-term CNCF commitment.
• OTel Profiles signal entered public alpha in March 2026 — continuous profiling is now the fourth pillar of observability alongside metrics, logs, and traces. This creates an opportunity to include AI workload profiling (GPU utilization patterns, inference latency profiling) in Morgan's observability extension.
• OTel is being adopted for AI/ML workload observability specifically — the ecosystem is expanding beyond traditional infrastructure monitoring to cover model inference, token throughput, and AI-specific golden signals.

Sources: CNCF OpenTelemetry Graduation announcement (prnewswire.com, May 2026), OTel Profiles Alpha blog post (opentelemetry.io, March 2026), OTel Graduated Now What blog (opentelemetry.io, 2026).

Updated Assessment

Dimension	Previous	Current	Delta
Feasibility	high	high	→
Impact	high	high	→
Urgency	med	high	↑ OTel graduation means the standard is stable and investment-safe. Building on it now avoids future migration costs.

Recommendation

Advance — OTel graduation removes the "alpha/beta risk" objection. The AI observability extension should be designed as an OTel-native workflow from the start, leveraging GenAI semantic conventions and the new Profiles signal for comprehensive AI workload visibility.