Add web-based API key management with secure backup and validation

[Copilot]

Implements a configuration UI for managing API keys, service toggles, cost limits, and pipeline settings through the dashboard. All changes persist to .env with automatic backups, validation, and rollback.

Changes

Backend (services/config_manager.py)

• ConfigManager service with atomic .env updates via python-dotenv.set_key()
• API key validation: regex patterns for OpenAI/Reddit/Gumroad, live testing against actual APIs
• Automatic timestamped backups (0o600 permissions) with 7-backup rotation
• Rollback on failure, complete audit trail with masked sensitive values

API Endpoints (dashboard.py)

GET  /api/config          # Returns config with masked API keys (e.g., "sk-...7890")
POST /api/config/update   # Validates, backs up, applies atomically
POST /api/config/test     # Tests API key validity (OpenAI/Reddit/Gumroad)
GET  /api/config/backups  # Lists timestamped backups
POST /api/config/restore  # Restores from backup

• Optional HTTP Basic Auth (DASHBOARD_PASSWORD) with auto_error=False for graceful degradation
• IP whitelisting middleware (DASHBOARD_ALLOWED_IPS)

Frontend UI (templates/config.html, static/)

• Dark theme configuration interface at /config
• API keys: masked inputs with visibility toggle, inline test buttons
• Service toggles: animated switches with Kill Switch confirmation dialog
• Cost/pipeline settings: validated numeric inputs
• Backup modal: list and restore previous configurations

Security

• File permissions: 0o600 on .env and all backups
• Value masking: sensitive data shows only first 3 + last 4 chars in UI and logs
• HTTPS warning: alerts on insecure connections (non-localhost/non-HTTPS)
• Atomic updates: write to temp file, then rename; rollback on error

Documentation

• Updated README.md with configuration management section
• Added DASHBOARD_PASSWORD and DASHBOARD_ALLOWED_IPS to .env.example
• Added config_backups/ to .gitignore
• New audit actions: config_updated, config_test, config_backup_created, config_restored

Screenshots

Configuration Interface

Backup Management

Configuration Schema

API_KEYS = {
    'OPENAI_API_KEY': r'^sk-[a-zA-Z0-9]{32,}$',
    'REDDIT_CLIENT_ID': r'^[a-zA-Z0-9_-]{14,}$',
    'REDDIT_CLIENT_SECRET': r'^[a-zA-Z0-9_-]{27,}$',
    'GUMROAD_ACCESS_TOKEN': r'^[a-zA-Z0-9_-]{32,}$',
}
COST_LIMITS = {
    'MAX_USD_PER_RUN': (0.01, 1000.0),
    'MAX_USD_LIFETIME': (1.0, 10000.0),
    'MAX_TOKENS_PER_RUN': (1000, 1000000),
}

Usage

python dashboard.py
# Navigate to http://localhost:8000/config

# Optional: Enable security in .env
DASHBOARD_PASSWORD=your_password
DASHBOARD_ALLOWED_IPS=127.0.0.1,192.168.1.0/24

Original prompt

🎯 Objective

Implement a comprehensive web-based API key management system that allows users to securely configure, test, and manage all API keys and service settings through the dashboard UI. All changes should persist to the .env file with automatic backups and robust validation.

---

📋 Requirements

1. Backend Service: services/config_manager.py

Create a new ConfigManager class with the following capabilities:

Configuration Structure

Manage 4 categories of settings:

API Keys:

• OPENAI_API_KEY - Pattern: ^sk-[a-zA-Z0-9]{32,}$ (required, masked)
• REDDIT_CLIENT_ID - Pattern: ^[a-zA-Z0-9_-]{14,}$ (required)
• REDDIT_CLIENT_SECRET - Pattern: ^[a-zA-Z0-9_-]{27,}$ (required, masked)
• REDDIT_USER_AGENT - Pattern: ^.{5,}$ (required)
• GUMROAD_ACCESS_TOKEN - Pattern: ^[a-zA-Z0-9_-]{32,}$ (required, masked)

Service Toggles (boolean):

• OPENAI_ENABLED - Enable/disable OpenAI calls
• REDDIT_ENABLED - Enable/disable Reddit ingestion
• GUMROAD_ENABLED - Enable/disable Gumroad publishing
• KILL_SWITCH - Emergency stop for entire pipeline

Cost Limits (numeric):

• MAX_USD_PER_RUN - Float, range: 0.01 to 1000.0
• MAX_USD_LIFETIME - Float, range: 1.0 to 10000.0
• MAX_TOKENS_PER_RUN - Integer, range: 1000 to 1000000

Pipeline Settings:

• REDDIT_SUBREDDITS - Comma-separated list, pattern: ^[a-zA-Z0-9_,-]+$
• REDDIT_MIN_SCORE - Integer, range: 0 to 100000
• REDDIT_POST_LIMIT - Integer, range: 1 to 100

Core Methods

class ConfigManager:
    def __init__(self, env_path: str = '.env')
    
    def get_current_config() -> Dict:
        """Return config with sensitive values masked (e.g., 'sk-...xyz')"""
    
    def update_config(updates: Dict, user_ip: str = None) -> Tuple[bool, List[str]]:
        """
        Validate inputs, create backup, apply changes atomically.
        On failure, rollback and restore from backup.
        Returns (success, errors_or_warnings)
        """
    
    def test_api_key(service: str, api_key: str) -> Tuple[bool, str]:
        """
        Test API key validity with actual API calls:
        - OpenAI: GET https://api.openai.com/v1/models
        - Reddit: Authenticate with praw
        - Gumroad: GET https://api.gumroad.com/v2/user
        Returns (is_valid, message)
        """
    
    def _validate_updates(updates: Dict):
        """
        Validate all inputs:
        - Check required fields are present
        - Validate regex patterns
        - Check numeric ranges
        - Raise ConfigValidationError on failure
        """
    
    def _create_backup() -> Path:
        """
        Create timestamped backup: config_backups/env_backup_YYYYMMDD_HHMMSS.txt
        Set permissions to 0o600 (owner read/write only)
        """
    
    def _restore_backup(backup_path: Path):
        """Restore .env from backup file"""
    
    def _apply_updates(updates: Dict):
        """
        Use python-dotenv's set_key() to write to .env file
        Convert booleans to 'true'/'false' strings
        """
    
    def _mask_value(value: str) -> str:
        """Mask sensitive values to show only first 3 and last 4 chars"""

Error Handling

• Create custom ConfigValidationError exception
• All file operations must be atomic (write to temp, then rename)
• Rollback on any failure during update
• Never log full API keys in audit trail

Security

• Set file permissions to 0o600 on .env and all backups
• Store backups in ./config_backups/ directory
• Keep last 7 backups, auto-delete older ones
• Mask sensitive values in all responses

---

2. Enhanced Dashboard API: dashboard.py

Add these new REST endpoints:

@app.get("/api/config")
def get_config(authenticated: bool = Depends(verify_dashboard_auth)):
    """Get current configuration with masked sensitive values"""

@app.post("/api/config/update")
async def update_config(request: Request, updates: dict, authenticated: bool = Depends(verify_dashboard_auth)):
    """
    Update configuration with validation.
    Request body format:
    {
        "api_keys": {"OPENAI_API_KEY": "sk-...", ...},
        "toggles": {"OPENAI_ENABLED": true, ...},
        "cost_limits": {"MAX_USD_PER_RUN": 5.0, ...},
        "pipeline": {"REDDIT_SUBREDDITS": "Entrepreneur,SaaS", ...}
    }
    """

@app.post("/api/config/test")
async def test_api_key(service: str, api_key: str, authenticated: bool = Depends(verify_dashboard_auth)):
    """Test API key validity before saving"""

@app.get("/api/config/backups")
def list_config_backups(authenticated: bool = Depends(verify_dashboard_auth)):
    """List all available configuration backups"""

@app.post("/api/config/restore")
async def restore_config_backup(backup_filename: str, authenticated: bool = Depends(verify_dashboard_auth)):
    """Restore configuration from a backup file"""

Security Middleware

Add optional HTTP Basic Auth:

from fastapi.security import HTTPBasic, HTTPBasicCredentia...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

*This pull request was created from Copilot chat.*
>

<!-- START COPILOT CODING AGENT TIPS -->
---

✨ Let Copilot coding agent [set things up for you](https://github.com/SaltProphet/Pi-autopilot/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo.