feat(web): Add roadmap gate metrics and MCP-first fallback policy

[LucasSantana-Dev]

Supersedes #451 (branch update blocked by repository rule GH013).

Summary

• add roadmap-aligned telemetry in GET /api/metrics for:
  • 50-user gate readiness (adoption.gate50)
  • core-flow adoption rates (onboarding/project/generation)
  • generation quality metrics (success, satisfaction, revisions)
  • MCP routing coverage (routing.mcp)
• enforce MCP-first generation path by default:
  • add ENABLE_MCP_DIRECT_PROVIDER_FALLBACK feature flag
  • default it to false
  • in MCP mode, return explicit error instead of silent direct-provider fallback unless explicitly enabled
• update MCP error guidance text to remove direct-provider assumption
• address Sonar reliability blockers:
  • generator mock includes yield-safe structure
  • routeViaMcp split into helper functions to reduce cognitive complexity below gate
• document behavior in README and CHANGELOG

Why

Roadmap priorities require validation/adoption metrics and completion of WebApp->Gateway integration with MCP as the primary flow before broader Phase 2 expansion.

Testing

• npm run lint
• npm run type-check
• npm run test
• npm run build
• targeted suites:
  • npm --workspace @siza/web test -- src/app/api/metrics/__tests__/route.test.ts src/__tests__/lib/services/provider-router.test.ts src/__tests__/lib/features/flags.test.ts

Summary by CodeRabbit

• New Features

  • Added direct provider fallback capability when MCP generation encounters failures, enabling more resilient generation workflows
  • Enhanced adoption and MCP routing metrics for better usage insights

• Improvements

  • Updated error messaging for MCP gateway unavailability to provide clearer health and configuration guidance

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
siza-web	Ready	Preview, Comment	Mar 11, 2026 8:17pm

[LucasSantana-Dev]

Code review (mandatory pass):

Findings

• No blocking findings.

Assumptions/Open questions

• MCP-first behavior is intentional for production paths: with ENABLE_MCP_GATEWAY=true and gateway configured, failures now return explicit MCP guidance unless ENABLE_MCP_DIRECT_PROVIDER_FALLBACK=true is explicitly enabled.
• /api/metrics expanded contract is consumed by internal dashboards/automation that can tolerate additive fields.

Summary

• Scope is focused and coherent: roadmap gate telemetry + MCP fallback policy + Sonar reliability compliance fixes.

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

Walkthrough

A new feature flag ENABLE_MCP_DIRECT_PROVIDER_FALLBACK enables MCP generation failures to fallback to direct provider calls. The flag is integrated into feature flag infrastructure, applied in provider routing logic, propagated through the API generation endpoint, tested across multiple modules, and metrics collection extended to track MCP-related adoption data.

Changes

Cohort / File(s)	Summary
Feature Flag Infrastructure apps/web/src/lib/features/flags.ts, apps/web/src/lib/features/types.ts, apps/web/src/__tests__/lib/features/flags.test.ts	Added new feature flag ENABLE_MCP_DIRECT_PROVIDER_FALLBACK to flag definitions, type union, and updated tests to expect 37 total flags with correct default values.
Provider Routing Logic apps/web/src/lib/services/provider-router.ts, apps/web/src/__tests__/lib/services/provider-router.test.ts	Added allowDirectProviderFallback?: boolean option to RouteGenerationOptions; implemented fallback helpers (resolveDirectProviderFallback, streamDirectProviderFallback) and modified MCP routing to conditionally fallback to direct provider when allowed; added test cases for both enabled and disabled fallback scenarios.
API Generation Endpoint apps/web/src/app/api/generate/route.ts	Added allowMcpDirectProviderFallback() helper to read the new flag; propagated allowDirectProviderFallback into routing pathway in POST and GET handlers; extended GET response features to conditionally include mcp-direct-provider-fallback capability.
Metrics Collection apps/web/src/app/api/metrics/route.ts	Added helper functions (asRows, toIdSet, roundRate, countActiveUsers, intersectionCount) and expanded database queries to fetch profiles, project owners, feedback, and MCP data; reworked aggregation logic to compute adoption metrics (gate50, onboarding, coreFlow) and routing.mcp metrics (total, last30d, coverageRate).
Metrics Tests apps/web/src/app/api/metrics/__tests__/route.test.ts	Expanded mock scaffolding with onboarding, project, feedback, and MCP metric data; updated expectations to verify new fields (revisions, satisfaction, adoption, routing.mcp) including zero-value behaviors when data is absent.
MCP Client apps/web/src/lib/mcp/client.ts	Updated error message in listTools from referencing fallback behavior to directing users to check gateway health and auth configuration.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant API as API Route<br/>(allowDirectProviderFallback)
    participant MCP as MCP Gateway
    participant Direct as Direct Provider
    
    Client->>API: POST /api/generate
    API->>API: Read ENABLE_MCP_DIRECT_PROVIDER_FALLBACK flag
    
    alt allowDirectProviderFallback = true
        API->>MCP: Route generation via MCP
        alt MCP Success
            MCP-->>API: Output stream
        else MCP Failure
            API->>Direct: Fallback: resolve default<br/>provider & model
            Direct-->>API: Stream generation events
        end
    else allowDirectProviderFallback = false
        API->>MCP: Route generation via MCP
        alt MCP Success
            MCP-->>API: Output stream
        else MCP Failure
            API-->>Client: Error response
        end
    end
    
    API-->>Client: Final response

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

• fix: Gateway routing code review fixes + provider-router tests #305: Modifies MCP fallback behavior in provider-router.ts, adjusting fallback event emission and fallback logic flow alongside this PR's policy-driven allowDirectProviderFallback implementation.
• fix: Improve generation API error messages #278: Overlaps on mcp/client.ts and generate/route.ts modifications, affecting MCP gateway error messaging and generate endpoint error handling paths.
• feat: Supabase-backed rate limiting + tighten free tier #372: Modifies provider routing logic in provider-router.ts, introducing changes to quota/fallback paths that intersect with this PR's new fallback routing flow.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title accurately reflects the two main objectives: adding roadmap gate metrics and implementing MCP-first fallback policy with the new feature flag.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch fix/roadmap-gate-metrics-sonar

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 OpenGrep (1.16.3)

apps/web/src/__tests__/lib/features/flags.test.ts

┌──────────────┐
│ Opengrep CLI │
└──────────────┘

�[32m✔�[39m �[1mOpengrep OSS�[0m
�[32m✔�[39m Basic security coverage for first-party code vulnerabilities.

�[1m Loading rules from local config...�[0m

apps/web/src/__tests__/lib/services/provider-router.test.ts

┌──────────────┐
│ Opengrep CLI │
└──────────────┘

�[32m✔�[39m �[1mOpengrep OSS�[0m
�[32m✔�[39m Basic security coverage for first-party code vulnerabilities.

�[1m Loading rules from local config...�[0m

apps/web/src/app/api/generate/route.ts

┌──────────────┐
│ Opengrep CLI │
└──────────────┘

�[32m✔�[39m �[1mOpengrep OSS�[0m
�[32m✔�[39m Basic security coverage for first-party code vulnerabilities.

�[1m Loading rules from local config...�[0m

• 6 others

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[github-actions]

Project Scorecard

Scorecard: 84/100 (B)
────────────────────────────────────────
  security: 100/100 (A)
  quality: 80/100 (B) — 1 violations
  performance: 67/100 (D) — 1 violations
  compliance: 75/100 (C) — 1 violations
  dependency: 100/100 (A)

Recommendations:
  - Increase test coverage to meet the 80% threshold
  - Extend log retention to at least 90 days for compliance