[STORY] OAuth 2.1 Authentication Infrastructure for MCP

[jsbattig]

Part of: #477

Story: OAuth Authentication for MCP Integration

Feature Group: F1 - OAuth 2.1 Authentication Infrastructure

📖 Story Overview

User Story

As a Claude.ai user, I want to authenticate with CIDX server using OAuth 2.1 so that I can securely access code search capabilities without managing API keys.

Objective

Implement a complete OAuth 2.1 authorization server that enables Claude and other MCP clients to authenticate securely with CIDX server, providing activity-based token management for seamless long-running sessions.

User Value

• 🔐 Secure authentication without API key management
• ⏱️ Long-lived sessions with automatic extension (8h base + activity)
• 🔄 Seamless token refresh without re-authentication
• 🎯 Standard OAuth flow familiar to developers

✅ Acceptance Criteria

Feature: OAuth 2.1 Authentication for MCP Clients

  Background:
    Given the CIDX server is running
    And OAuth endpoints are configured

  Scenario: Discover OAuth endpoints
    When a client requests "/.well-known/oauth-authorization-server"
    Then the response contains authorization endpoint "/oauth/authorize"
    And the response contains token endpoint "/oauth/token"
    And the response contains registration endpoint "/oauth/register"
    And the response indicates PKCE is required
    And supported grant types include "authorization_code"

  Scenario: Dynamic client registration
    When Claude sends a registration request to "/oauth/register"
    With client name "Claude.ai MCP Client"
    And redirect URI "https://claude.ai/oauth/callback"
    Then a client_id is generated and returned
    And the client is stored in OAuth database
    And the response includes client_id and registration metadata

  Scenario: Authorization code flow with PKCE
    Given a registered OAuth client exists
    When the client initiates authorization with:
      | client_id       | generated_client_id |
      | redirect_uri    | registered_uri      |
      | code_challenge  | valid_pkce_challenge|
      | response_type   | code                |
      | state          | random_state        |
    Then user is redirected to login page
    When user successfully authenticates
    Then authorization code is generated
    And user is redirected to callback with code and state

  Scenario: Token exchange with PKCE verification
    Given a valid authorization code exists
    When client exchanges code for token with:
      | grant_type     | authorization_code  |
      | code          | valid_auth_code     |
      | code_verifier | matching_verifier   |
      | client_id     | registered_client   |
    Then access token is generated with 8-hour expiration
    And refresh token is generated with 30-day expiration
    And tokens are associated with authenticated user
    And response includes token_type "Bearer"

  Scenario: Activity-based token extension
    Given a valid access token with 3 hours remaining
    When the token is used for an API request
    Then the token expiration extends by 8 hours
    And the new expiration is capped at 30 days from creation
    And the extension is recorded in the database

  Scenario: Token validation for MCP requests
    Given a request to "/mcp" endpoint
    When the request includes "Authorization: Bearer <token>"
    Then the token is validated against OAuth storage
    And user context is extracted from token
    And permissions are loaded from UserManager
    And request proceeds if token is valid

  Scenario: Handle expired tokens
    Given an expired access token
    When the token is used for an API request
    Then the response returns 401 Unauthorized
    And error indicates "token_expired"
    And client must use refresh token or re-authenticate

🔧 Technical Requirements

Database Schema (SQLite)

-- pseudocode schema
TABLE oauth_clients (
    client_id TEXT PRIMARY KEY,
    client_name TEXT NOT NULL,
    redirect_uris JSON NOT NULL,
    created_at TIMESTAMP,
    metadata JSON
)

TABLE oauth_codes (
    code TEXT PRIMARY KEY,
    client_id TEXT REFERENCES oauth_clients,
    user_id TEXT NOT NULL,
    code_challenge TEXT NOT NULL,
    redirect_uri TEXT NOT NULL,
    expires_at TIMESTAMP,
    used BOOLEAN DEFAULT FALSE
)

TABLE oauth_tokens (
    token_id TEXT PRIMARY KEY,
    client_id TEXT REFERENCES oauth_clients,
    user_id TEXT NOT NULL,
    access_token TEXT UNIQUE NOT NULL,
    refresh_token TEXT UNIQUE,
    expires_at TIMESTAMP,
    created_at TIMESTAMP,
    last_activity TIMESTAMP,
    hard_expires_at TIMESTAMP
)

API Endpoints

# pseudocode endpoints
GET /.well-known/oauth-authorization-server
    returns: OAuth discovery metadata

POST /oauth/register
    body: {client_name, redirect_uris, grant_types}
    returns: {client_id, client_secret_expires_at: 0}

GET /oauth/authorize
    params: client_id, redirect_uri, response_type, code_challenge, state
    returns: redirect to login or consent page

POST /oauth/token
    body: {grant_type, code?, code_verifier?, refresh_token?}
    returns: {access_token, token_type, expires_in, refresh_token}

Token Management Logic

# pseudocode for activity extension
def extend_token_on_activity(token):
    remaining = token.expires_at - now()
    if remaining < 4_hours:
        new_expiry = now() + 8_hours
        max_expiry = token.created_at + 30_days
        token.expires_at = min(new_expiry, max_expiry)
        token.last_activity = now()
        save_token(token)

Integration Points

• UserManager: Validate credentials during authorization
• FastAPI: Mount OAuth routes in main app
• Auth Middleware: Extend to support Bearer tokens
• SQLite Storage: Atomic transactions for token operations

📋 Implementation Checklist

Setup & Configuration

• Install authlib>=1.3.0 and aiosqlite>=0.19.0
• Create OAuth module at /src/code_indexer/server/auth/oauth/
• Initialize SQLite database at ~/.cidx-server/oauth.db
• Configure OAuth server settings (issuer URL, algorithms)

OAuth Server Implementation

• Implement OAuthManager class with Authlib
• Create SQLite storage layer with async operations
• Implement discovery endpoint (.well-known)
• Implement dynamic client registration
• Implement authorization endpoint with login integration
• Implement token endpoint with PKCE validation
• Implement token introspection (optional but recommended)

Security Implementation

• Enforce PKCE for all authorization flows
• Generate cryptographically secure tokens (secrets.token_urlsafe)
• Implement CSRF protection for authorization
• Add rate limiting for token endpoints
• Implement secure token storage with hashing

Token Management

• Implement 8-hour initial expiration
• Implement activity-based extension logic
• Implement 30-day hard expiration limit
• Create token cleanup job for expired tokens
• Add token revocation endpoint

Integration

• Modify auth dependencies to support Bearer tokens
• Create unified auth middleware for JWT/OAuth
• Update UserManager integration
• Mount OAuth routes in FastAPI app
• Update CORS settings for OAuth redirects

🧪 Testing Requirements

Unit Tests

• Test OAuth discovery endpoint
• Test client registration validation
• Test PKCE generation and validation
• Test authorization code generation
• Test token exchange with various grant types
• Test activity-based extension logic
• Test token expiration boundaries

Integration Tests

• Test complete OAuth flow end-to-end
• Test Bearer token validation in API calls
• Test token extension during API activity
• Test expired token handling
• Test refresh token flow
• Test concurrent token operations

Security Tests

• Test PKCE requirement enforcement
• Test authorization code replay prevention
• Test token uniqueness
• Test SQL injection prevention
• Test CSRF protection

Performance Tests

• Test token validation speed (<5ms)
• Test concurrent authorization flows
• Test database lock handling
• Test token cache effectiveness

📝 Definition of Done

• All OAuth 2.1 endpoints implemented and functional
• PKCE validation working and required
• Activity-based token extension working (8h + activity, 30d max)
• SQLite storage layer operational with migrations
• Integration with existing UserManager complete
• Bearer token validation in auth middleware
• All unit tests passing (>90% coverage)
• Integration tests passing with Claude.ai simulator
• Security audit completed (OWASP compliance)
• Performance targets met (<5ms token validation)
• Documentation complete (setup, configuration, API reference)
• Error handling comprehensive with proper status codes
• Logging implemented for security events
• Database cleanup job scheduled

---

Story Metadata:

• Type: Feature
• Priority: Critical (BLOCKER for other stories)
• Points: 8
• Components: Authentication, OAuth, Security
• Labels: story, oauth, authentication, mcp-integration
• Dependencies: None (foundational)
• Blocked By: None
• Blocks: All other MCP stories

[jsbattig]

🔍 Implementation Review - Existing Infrastructure (2025-11-14)

✅ SIGNIFICANT INFRASTRUCTURE ALREADY IN PLACE

Based on comprehensive code review, CIDX server already has advanced token management infrastructure that reduces OAuth 2.1 implementation complexity:

---

🏗️ Existing Token Management Infrastructure

1. Refresh Token System (PRODUCTION-READY)

Location: src/code_indexer/server/auth/refresh_token_manager.py (659 lines)

Features Implemented ✅:

• Token Family Tracking: Multi-generation token relationships in SQLite
• Replay Attack Detection: Automatic family revocation on suspicious activity
• Concurrent Refresh Protection: Thread-safe operations with locks
• SQLite Persistence: ~/.cidx-server/refresh_tokens.db with atomic operations
• Secure Storage: SHA256 token hashing (never stores plaintext)
• Configurable Lifetime: 7-day default, cleanup jobs implemented
• Audit Integration: Comprehensive security event logging

Database Tables ✅:

-- ALREADY EXISTS
TABLE token_families (
    family_id TEXT PRIMARY KEY,
    username TEXT NOT NULL,
    created_at TEXT,
    last_used_at TEXT,
    is_revoked INTEGER,
    revocation_reason TEXT
)

TABLE refresh_tokens (
    token_id TEXT PRIMARY KEY,
    family_id TEXT NOT NULL,
    username TEXT NOT NULL,
    token_hash TEXT NOT NULL,  -- SHA256, not plaintext
    created_at TEXT,
    expires_at TEXT,
    is_used INTEGER,
    used_at TEXT,
    parent_token_id TEXT
)

OAuth Integration Benefit: Can use same proven patterns for OAuth refresh tokens.

---

2. Rate Limiting Infrastructure (PRODUCTION-READY)

Location: src/code_indexer/server/auth/rate_limiter.py

Features Implemented ✅:

• Per-user rate limiting (5 attempts, 15-min lockout)
• Automatic cleanup of expired entries
• Thread-safe with lock protection
• Configurable thresholds
• Currently applies to password changes

OAuth Integration: Can rate-limit OAuth token requests, authorization attempts.

---

3. Comprehensive Audit Logging (PRODUCTION-READY)

Location: src/code_indexer/server/auth/audit_logger.py

Events Logged ✅:

• Authentication attempts (success/failure)
• Password changes with reasons
• Token refresh operations
• Security incidents (replay attacks, rate limits)
• Admin operations
• Permission denials

Log Format:

AuditLogEntry {
    timestamp: datetime
    username: str
    event_type: str
    status: str  # "success", "failure", "security_incident"
    ip_address: str
    additional_context: Dict[str, Any]
}

OAuth Integration: OAuth events (authorization, token exchange, refresh) fit naturally into existing framework.

---

4. Timing Attack Prevention (PRODUCTION-READY)

Location: Password comparison logic in password_manager.py

Protection ✅:

• Constant-time password comparisons
• Prevents username enumeration
• Prevents token validation timing leaks

OAuth Integration: Apply same patterns to OAuth client secret validation.

---

5. JWT Token Manager (PRODUCTION-READY)

Location: src/code_indexer/server/auth/jwt_manager.py

Features Implemented ✅:

• Token creation with HS256 algorithm
• Activity-based expiration extension (extend_token_expiration())
• High-precision timestamps (microsecond)
• JWT ID (jti) for blacklist support
• Configurable expiration (10-min default)

OAuth Integration: Activity extension pattern can be adapted for OAuth tokens.

---

📋 Updated Implementation Checklist

Setup & Configuration

• Install authlib>=1.3.0 - TODO: Add to dependencies
• Install aiosqlite>=0.19.0 - TODO: Add to dependencies
• Create OAuth module at /src/code_indexer/server/auth/oauth/
• Initialize OAuth database at ~/.cidx-server/oauth.db (pattern exists in refresh_tokens.db)
• Configure OAuth server settings (issuer URL, algorithms)

OAuth Server Implementation

• Implement OAuthManager class with Authlib
• Create SQLite storage layer (PATTERN EXISTS: Use refresh_token_manager.py as template)
• Implement discovery endpoint (.well-known)
• Implement dynamic client registration
• Implement authorization endpoint with login integration
• Implement token endpoint with PKCE validation
• Implement token introspection (optional)

Security Implementation

• Enforce PKCE for all authorization flows
• Generate cryptographically secure tokens - EXISTS: secrets.token_urlsafe() used in refresh tokens
• Implement CSRF protection for authorization
• Add rate limiting for token endpoints - LEVERAGE EXISTING: Rate limiter already implemented
• Implement secure token storage with hashing - LEVERAGE EXISTING: SHA256 pattern exists in refresh tokens

Token Management

• Implement 8-hour initial expiration
• Implement activity-based extension logic - PATTERN EXISTS: JWT has extend_token_expiration()
• Implement 30-day hard expiration limit
• Create token cleanup job for expired tokens - PATTERN EXISTS: cleanup_expired_tokens() in refresh manager
• Add token revocation endpoint

Integration

• Modify auth dependencies to support Bearer tokens
• Create unified auth middleware for JWT/OAuth
• Update UserManager integration - EXISTS: Full user management with roles
• Mount OAuth routes in FastAPI app
• Update CORS settings for OAuth redirects

---

💡 Implementation Recommendations

1. Leverage Refresh Token Patterns

The existing refresh token system demonstrates best practices:

• Use the same SQLite storage approach
• Adopt the same token family tracking pattern
• Reuse the cleanup job pattern
• Apply the same audit logging integration

Estimated Effort Reduction: 30-40% (existing patterns proven at scale)

###2. Reuse Security Infrastructure
Don't rebuild what exists:

• Use existing rate limiter for OAuth endpoints
• Use existing audit logger for OAuth events
• Use existing timing-safe comparison patterns
• Use existing token hashing patterns (SHA256)

Estimated Effort Reduction: 20-30% (security infrastructure ready)

3. Unified Token Management

Consider creating TokenManager that coordinates:

• JWT tokens (short-lived, stateless)
• OAuth Bearer tokens (8h + activity, stateful)
• Refresh tokens (7-day, family tracked)

All using shared infrastructure for storage, auditing, rate limiting.

---

🎯 Adjusted Scope

What Needs Building (NEW)

• OAuth discovery endpoint
• OAuth client registration
• OAuth authorization endpoint
• OAuth token endpoint
• PKCE validation logic
• OAuth-specific database tables

What Can Be Leveraged (EXISTING)

• ✅ Token family tracking pattern
• ✅ Replay attack detection
• ✅ Concurrent operation protection
• ✅ SQLite persistence with atomic operations
• ✅ Rate limiting infrastructure
• ✅ Audit logging framework
• ✅ Timing attack prevention
• ✅ Activity-based extension pattern
• ✅ Token cleanup jobs

---

📊 Complexity Assessment

Original Estimate: 8 points (full OAuth implementation from scratch)
Adjusted Estimate: 5-6 points (with existing infrastructure)

Reasoning:

• 40% of token management infrastructure exists
• Security infrastructure is production-ready
• Proven patterns reduce implementation risk
• Primary work is OAuth-specific protocol layer

---

🚀 Next Steps

1. Review existing refresh token implementation (refresh_token_manager.py)
2. Design OAuth tables using same patterns
3. Implement OAuth manager using Authlib
4. Integrate with existing security infrastructure
5. Add OAuth routes to FastAPI app
6. Test with Claude.ai OAuth flow

Priority: CRITICAL - Blocks all other MCP stories

---

Review Date: 2025-11-14
Infrastructure Status: 40% complete (token management patterns exist)
Implementation Risk: REDUCED (proven patterns, production-tested security)