feat(mcp): add structured metadata field to MCP tool responses

[prosdev]

🎯 Problem

Currently, MCP tool responses embed operational metadata (like token counts) within the markdown content string:

{
  "action": "search",
  "content": "## Results...\n\n🪙 ~31 tokens"
}

Issues with current approach:

• Token counts require string parsing to extract
• No separation between user-facing content and operational metadata
• Difficult to aggregate metrics across calls
• Not machine-readable for optimization tracking
• Cannot easily add new metadata without polluting content

🎯 Goal

Token counts are critical metadata for tracking optimization and adoption pain points. We need a structured way to expose this data to enable:

• Performance monitoring and optimization tracking
• Token reduction measurement
• Client-side analytics and dashboards
• Data quality and freshness indicators
• Informed decision-making about search strategies

💡 Proposed Solution

Add a structured metadata field to all MCP tool responses with three tiers of implementation:

Tier 1: Initial Implementation (Core Metrics)

interface MCPMetadata {
  // Cost tracking
  tokens: number;                    // Approximate token count (documented)
  
  // Performance
  duration_ms: number;               // Response time for optimization
  timestamp: string;                 // ISO 8601 timestamp
  
  // Data freshness
  cached: boolean;                   // Was response from cache?
  
  // Data quality
  results_total?: number;            // Total matches found
  results_returned?: number;         // Results in this response
  results_truncated?: boolean;       // Were results cut off?
}

Example response:

{
  "action": "search",
  "query": "logger integration",
  "format": "compact",
  "content": "## GitHub Search Results\n**Query:** \"logger integration\"\n**Found:** 1 results\n- 🔵 × #6: Implement CLI and Integration Examples [18%]",
  "metadata": {
    "tokens": 31,
    "duration_ms": 142,
    "timestamp": "2025-11-25T10:30:00Z",
    "cached": false,
    "results_total": 1,
    "results_returned": 1,
    "results_truncated": false
  }
}

Tier 2: Near-Term Enhancements (Quality Indicators)

interface MCPMetadata {
  // ... Tier 1 fields ...
  
  // Search quality
  confidence?: number;               // 0-1 relevance score for semantic search
  threshold?: number;                // Search threshold applied
  
  // Index information
  index_size?: number;               // Total items in searchable corpus
  index_last_updated?: string;       // When index was last refreshed
  
  // Issues & warnings
  warnings?: Array<{
    code: string;                    // e.g., "INDEX_STALE"
    message: string;                 // Human-readable message
    severity: "info" | "warning" | "error";
  }>;
}

Example with warnings:

{
  "metadata": {
    "tokens": 31,
    "duration_ms": 142,
    "cached": true,
    "index_last_updated": "2025-11-23T08:00:00Z",
    "warnings": [
      {
        "code": "INDEX_STALE",
        "message": "Index is 2 days old, run 'dev-agent update'",
        "severity": "warning"
      }
    ]
  }
}

Tier 3: Future Enhancements (Debugging & Advanced)

interface MCPMetadata {
  // ... Tier 1 & 2 fields ...
  
  // Debugging
  request_id?: string;               // For log correlation
  tool_version?: string;             // MCP server version
  adapter?: string;                  // Which adapter handled request
  
  // Resource usage
  tokens_input?: number;             // Tokens in query
  tokens_output?: number;            // Tokens in response
  api_calls_made?: number;           // External API calls
  rate_limit_remaining?: number;     // GitHub/external API limits
  
  // Tool-specific context
  tool_context?: Record<string, unknown>;  // Flexible tool-specific data
}

📋 Implementation Tasks

Phase 1: Core Infrastructure (Tier 1)

1.1 Type Definitions

• Define MCPMetadata interface in packages/mcp-server/src/server/types.ts
• Update MCPResponse interfaces to include metadata field
• Create utility types for metadata building

1.2 Update Formatters

Change all formatters to return structured data:

• packages/mcp-server/src/formatters/search.ts
• packages/mcp-server/src/formatters/github.ts
• packages/mcp-server/src/formatters/plan.ts
• packages/mcp-server/src/formatters/status.ts
• packages/mcp-server/src/formatters/explore.ts

Return format:

return {
  content: formattedString,
  tokens: calculatedTokens  // Extract from current string format
};

1.3 Update Adapters

All adapters construct metadata:

• github-adapter.ts - Add timing, result counts
• search-adapter.ts - Add timing, result counts, confidence
• plan-adapter.ts - Add timing, tasks generated
• status-adapter.ts - Add timing, index info
• explore-adapter.ts - Add timing, files analyzed

1.4 Remove Token Display from Content

• Remove 🪙 ~XX tokens suffix from all formatter outputs
• Clean up content formatting

1.5 Testing

• Update all adapter tests to expect metadata field
• Add tests for metadata structure validation
• Verify token counts match previous display
• Add integration tests for timing accuracy

Phase 2: Quality Enhancements (Tier 2)

• Add confidence scoring to semantic search results
• Track and expose index freshness
• Implement warning system for stale data
• Add threshold information to search metadata

Phase 3: Advanced Features (Tier 3)

• Add request ID generation and tracking
• Implement detailed token breakdowns
• Add rate limit tracking for external APIs
• Create tool-specific context fields

Documentation

• Document that token counts are approximate
• Update README with new response format and examples
• Add migration guide for breaking change
• Document all metadata fields with examples per tool
• Update CURSOR_SETUP.md with metadata usage examples

🔧 Tool-Specific Examples

dev_search

{
  "action": "search",
  "metadata": {
    "tokens": 45,
    "duration_ms": 230,
    "cached": false,
    "results_total": 12,
    "results_returned": 10,
    "confidence": 0.82,
    "threshold": 0.7,
    "index_size": 1250
  }
}

dev_gh

{
  "action": "context",
  "metadata": {
    "tokens": 113,
    "duration_ms": 85,
    "cached": true,
    "index_last_updated": "2025-11-25T08:00:00Z",
    "tool_context": {
      "github_repo": "lytics/dev-agent",
      "issue_number": 6,
      "has_comments": true
    }
  }
}

dev_plan

{
  "action": "plan",
  "metadata": {
    "tokens": 450,
    "duration_ms": 1200,
    "cached": false,
    "tool_context": {
      "issue_number": 29,
      "related_files_analyzed": 15,
      "tasks_generated": 8,
      "detail_level": "detailed",
      "used_explorer": true
    }
  }
}

dev_explore

{
  "action": "pattern",
  "metadata": {
    "tokens": 280,
    "duration_ms": 450,
    "cached": false,
    "results_total": 24,
    "results_returned": 10,
    "confidence": 0.75,
    "tool_context": {
      "action": "pattern",
      "patterns_found": 12,
      "files_analyzed": 45
    }
  }
}

dev_status

{
  "action": "status",
  "metadata": {
    "tokens": 156,
    "duration_ms": 45,
    "cached": true,
    "index_size": 1250,
    "index_last_updated": "2025-11-25T08:00:00Z",
    "warnings": [
      {
        "code": "INDEX_STALE",
        "message": "Index is 2 days old",
        "severity": "warning"
      }
    ]
  }
}

✅ Benefits

1. Machine Readable: No string parsing needed
2. Type Safe: Structured data with proper TypeScript types
3. Extensible: Three-tier approach allows gradual enhancement
4. Standard Pattern: Follows industry best practices (GitHub, Stripe, GraphQL)
5. Optimization Tracking: Easy to build analytics and monitoring
6. Client Flexibility: Clients choose how/if to display metadata
7. Data Quality: Users understand freshness and completeness
8. Performance Insights: Track and optimize slow operations

🚨 Breaking Change Notice

This is a breaking change to the MCP response format.

Impact:

• Clients parsing token counts from content string need updates
• Response structure changes (adds metadata field)

Migration Path:

1. Read from metadata.tokens instead of parsing content
2. Use metadata.results_* fields for result information
3. Content will be cleaner without token/result suffixes
4. Clients can optionally display additional metadata

Timeline:

• Phase 1 (Tier 1): Next minor version (breaking)
• Phase 2 (Tier 2): Non-breaking additions to metadata
• Phase 3 (Tier 3): Non-breaking additions to metadata

📊 Success Metrics

After Phase 1 implementation:

• ✅ Track token reduction across tool improvements
• ✅ Identify which tools/queries are most expensive
• ✅ Build dashboards showing token usage trends
• ✅ Make data-driven decisions about optimization priorities
• ✅ Monitor response time performance
• ✅ Track cache hit rates

After Phase 2:

• ✅ Surface data freshness issues proactively
• ✅ Help users understand search quality
• ✅ Reduce support burden with better warnings

After Phase 3:

• ✅ Full observability for debugging
• ✅ Rate limit awareness and management
• ✅ Detailed cost attribution

🎨 Design Decisions

Why metadata vs top-level fields?

• Extensible for future fields without API changes
• Semantic clarity - metadata about the response
• Industry standard pattern
• Groups related operational data

Remove tokens from content?

Yes - cleaner separation, no duplication, client control

Token precision?

Whole numbers documented as approximate - simpler and sufficient

Tiered approach?

Yes - ship incrementally, validate each tier, avoid over-engineering

Tool-specific vs uniform?

Hybrid - common base fields + optional tool_context for specific data

🔗 Related

• Discovered during testing after PR fix: replace dynamic import with static import for GitHubIndexer #50
• Part of broader MCP server improvements
• Supports adoption and optimization goals
• Enables future analytics and monitoring features