Add web-based configuration management system with dark theme, hardware monitoring, and data visualization

[Copilot]

Eliminates need for SSH access to edit .env files. Users can now manage API keys, service toggles, cost limits, and pipeline settings through a web UI with validation, testing, and automatic backup/restore. Features a modern dark theme with real-time hardware monitoring and data visualization.

Changes

Backend (services/config_manager.py)

• ConfigManager service: Handles .env CRUD operations with atomic writes and 0o600 permissions
• Validation engine: Regex patterns for API keys, range checks for numeric fields, type validation
• API key testing: Direct validation against OpenAI, Reddit (praw), and Gumroad APIs before saving
• Backup system: Timestamped backups on every write, keeps last 7, restore capability with rollback on failure
• Value masking: Shows first 3 + last 4 chars of sensitive fields (e.g., sk-***...7890)

API Endpoints (dashboard.py)

GET  /config                  # Serve config UI
GET  /api/config              # Current config with masked values
POST /api/config/update       # Validate and apply changes
POST /api/config/test         # Test API key before saving
GET  /api/config/backups      # List available backups
POST /api/config/restore      # Restore from backup
GET  /api/hardware            # System hardware metrics (NEW)

Security: Optional HTTP Basic Auth (DASHBOARD_PASSWORD), IP whitelisting (DASHBOARD_ALLOWED_IPS)

Frontend

Dark Theme UI

• Modern dark color scheme: #0f0f23 background with blue (#6c9ef8) accents
• Glassmorphism effects: Semi-transparent cards with subtle borders
• Improved contrast: Light text (#e0e0e0) on dark backgrounds for optimal readability
• Consistent styling: All components redesigned to match dark theme aesthetic

Hardware Monitoring System

• Real-time metrics (updates every 3 seconds):
  • 🖥️ CPU Usage with percentage display
  • 💾 Memory Usage showing used/total GB
  • 💿 Disk Usage showing used/total GB
  • 🌡️ CPU Temperature in Celsius (when available)
• Color-coded progress bars:
  • Green: Normal operation
  • Yellow: Warning threshold
  • Red: Critical levels
• Cross-platform support: Works on Windows, Linux, and macOS using psutil

Data Visualization

• Chart.js integration: Real-time line graphs showing system trends
• Historical tracking: CPU and Memory usage over last 20 data points
• Interactive tooltips: Hover to see detailed metrics
• Smooth updates: Efficient rendering without animations for better performance

Interactive Features

• Responsive UI (templates/config.html): Five sections (System Monitoring, API Keys, Service Toggles, Cost Limits, Pipeline Settings)
• Test buttons: Validate API keys before saving
• Show/hide toggles: Secure password field visibility controls
• Visual switches: Modern toggle switches for boolean values
• Toast notifications: Real-time feedback for user actions
• Loading states: Visual indicators during async operations
• Confirmation dialogs: Safety prompts for destructive actions
• Security warning: Banner shown when accessing over HTTP from non-localhost

Configuration Schema

CONFIG_FIELDS = {
    'api_keys': {
        'OPENAI_API_KEY': {
            'pattern': r'^sk-[a-zA-Z0-9_-]{32,}$',
            'masked': True,
            'test_url': 'https://api.openai.com/v1/models'
        },
        # ... Reddit, Gumroad keys
    },
    'toggles': {
        'OPENAI_ENABLED': {'type': 'boolean', 'default': True},
        'KILL_SWITCH': {'type': 'boolean', 'warning': '...'}
    },
    'cost_limits': {
        'MAX_USD_PER_RUN': {'type': 'float', 'min': 0.01, 'max': 1000.0}
    },
    'pipeline': {
        'REDDIT_SUBREDDITS': {'pattern': r'^[a-zA-Z0-9_,-]+$'}
    }
}

Updated Configuration Support

• New toggles: OPENAI_ENABLED, REDDIT_ENABLED, GUMROAD_ENABLED (in addition to existing KILL_SWITCH)
• Security settings: DASHBOARD_PASSWORD, DASHBOARD_ALLOWED_IPS
• Audit log: New actions for config_updated, config_test, config_backup_created, config_restored

Dependencies Added

• psutil>=5.9.0 - For cross-platform hardware monitoring
• Chart.js 4.4.0 (CDN) - For real-time data visualization

Screenshots

Dark Theme with Hardware Monitoring:

Testing

16 unit tests covering validation, backup/restore, API key testing, masking logic. All passing.

Features Summary

Configuration Management:

• ✅ Web-based .env editing without SSH access
• ✅ API key testing before saving
• ✅ Automatic backups with restore capability
• ✅ Service toggles and cost limit controls
• ✅ Comprehensive input validation

User Interface:

• ✅ Modern dark theme with glassmorphism effects
• ✅ Real-time hardware monitoring dashboard
• ✅ Interactive data visualization charts
• ✅ Responsive design for mobile and desktop
• ✅ Color-coded health indicators

Security:

• ✅ Optional HTTP Basic Auth
• ✅ IP whitelisting support
• ✅ Secure file permissions (0o600)
• ✅ Value masking for sensitive data
• ✅ Complete audit trail

Original prompt

API Key Management System - Implementation Specification

Objective

Create a secure, web-based configuration management system that allows users to input and manage API keys, toggle services on/off, and modify configuration settings through a GUI, with automatic .env file updates and robust error handling.

Core Requirements

1. Backend Service: services/config_manager.py

Create a new configuration management service with the following capabilities:

Class: ConfigManager

• Manage .env file operations (read, write, backup, restore)
• Validate all configuration inputs before applying
• Create timestamped backups before any changes
• Support rollback on failure
• Audit log all configuration changes
• Secure file permissions (0o600 for .env files)

Configuration Structure:

CONFIG_FIELDS = {
    'api_keys': {
        'OPENAI_API_KEY': {
            'label': 'OpenAI API Key',
            'pattern': r'^sk-[a-zA-Z0-9]{32,}$',
            'required': True,
            'masked': True,
            'test_url': 'https://api.openai.com/v1/models',
        },
        'REDDIT_CLIENT_ID': {
            'label': 'Reddit Client ID',
            'pattern': r'^[a-zA-Z0-9_-]{14,}$',
            'required': True,
            'masked': False,
        },
        'REDDIT_CLIENT_SECRET': {
            'label': 'Reddit Client Secret',
            'pattern': r'^[a-zA-Z0-9_-]{27,}$',
            'required': True,
            'masked': True,
        },
        'REDDIT_USER_AGENT': {
            'label': 'Reddit User Agent',
            'pattern': r'^.{5,}$',
            'required': True,
            'masked': False,
        },
        'GUMROAD_ACCESS_TOKEN': {
            'label': 'Gumroad Access Token',
            'pattern': r'^[a-zA-Z0-9_-]{32,}$',
            'required': True,
            'masked': True,
        },
    },
    'toggles': {
        'OPENAI_ENABLED': {'label': 'Enable OpenAI', 'type': 'boolean', 'default': True},
        'REDDIT_ENABLED': {'label': 'Enable Reddit Ingestion', 'type': 'boolean', 'default': True},
        'GUMROAD_ENABLED': {'label': 'Enable Gumroad Publishing', 'type': 'boolean', 'default': True},
        'KILL_SWITCH': {
            'label': 'Kill Switch (Stop All Pipeline)',
            'type': 'boolean',
            'default': False,
            'warning': 'Enabling this will stop all pipeline execution!',
        },
    },
    'cost_limits': {
        'MAX_USD_PER_RUN': {'label': 'Max USD per Run', 'type': 'float', 'min': 0.01, 'max': 1000.0, 'default': 5.0},
        'MAX_USD_LIFETIME': {'label': 'Max USD Lifetime', 'type': 'float', 'min': 1.0, 'max': 10000.0, 'default': 100.0},
        'MAX_TOKENS_PER_RUN': {'label': 'Max Tokens per Run', 'type': 'integer', 'min': 1000, 'max': 1000000, 'default': 50000},
    },
    'pipeline': {
        'REDDIT_SUBREDDITS': {'label': 'Subreddits (comma-separated)', 'type': 'string', 'pattern': r'^[a-zA-Z0-9_,-]+$', 'default': 'Entrepreneur,SaaS,startups'},
        'REDDIT_MIN_SCORE': {'label': 'Minimum Reddit Score', 'type': 'integer', 'min': 0, 'max': 100000, 'default': 10},
        'REDDIT_POST_LIMIT': {'label': 'Posts per Run', 'type': 'integer', 'min': 1, 'max': 100, 'default': 25},
    },
}

Key Methods:

• get_current_config() - Retrieve current config with masked sensitive values
• update_config(updates: Dict, user_ip: str) - Validate and apply updates with backup
• test_api_key(service: str, api_key: str) - Test API key validity
• _validate_updates(updates: Dict) - Validate all inputs against patterns/ranges
• _create_backup() - Create timestamped backup in ./config_backups/
• _restore_backup(backup_path) - Restore from backup file
• _mask_value(value: str) - Mask sensitive values (show first 3 and last 4 chars)
• _apply_updates(updates: Dict) - Write to .env using python-dotenv.set_key()

API Key Testing:

• _test_openai_key() - HTTP GET to OpenAI API with Authorization header
• _test_reddit_credentials() - Use praw to authenticate
• _test_gumroad_key() - HTTP GET to Gumroad API

Error Handling:

• Custom exception: ConfigValidationError
• Rollback on any failure during update
• Detailed error messages for validation failures
• Audit logging of all changes and failures

2. Enhanced Dashboard API: dashboard.py

Add the following REST API endpoints:

# GET /api/config
# Returns current configuration with masked sensitive values
# Response: {"success": True, "config": {...}}

# POST /api/config/update
# Updates configuration with validation
# Request body: {"api_keys": {...}, "toggles": {...}, "cost_limits": {...}, "pipeline": {...}}
# Response: {"success": True, "message": "...", "warnings": [...]}

# POST /api/config/test
# Tests API key before saving
# Query params: service (OPENAI/REDDIT/GUMROAD), api_key
# Response: {"success": True/False, "message": "..."}

# GET /api/config/backups
# Lists all configuration backups
# Response: {"success": True, "backups": [{...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

*This pull request was created from Copilot chat.*
>

<!-- START COPILOT CODING AGENT TIPS -->
---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey).

[SaltProphet]

@copilot implment UI changes to a dark color scheme and add hardware onitoring and data visualization monitoring.

[Copilot]

@copilot implment UI changes to a dark color scheme and add hardware onitoring and data visualization monitoring.

Implemented in commit 3d9a1e7:

✅ Dark color scheme - Complete UI redesign with #0f0f23 background, dark cards, and #6c9ef8 blue accents
✅ Hardware monitoring - Real-time CPU, memory, disk usage, and temperature tracking (updates every 3s)
✅ Data visualization - Chart.js integration showing CPU/Memory trends with interactive graphs

The configuration page now features:

• 🎨 Modern dark theme with glassmorphism effects
• 📊 System Monitoring section with 4 hardware metric cards
• 📈 Live charts displaying CPU and memory usage history
• 🚦 Color-coded progress bars (green/yellow/red based on thresholds)

New API endpoint: GET /api/hardware returns real-time system stats using psutil.

[SaltProphet]

@copilot implement changes

[Copilot]

@copilot implement changes

Could you please clarify what specific changes you'd like me to implement?

I've already completed:

• ✅ Dark theme UI redesign
• ✅ Hardware monitoring (CPU, Memory, Disk, Temperature)
• ✅ Data visualization with Chart.js

If you need additional features or modifications, please let me know what you'd like changed.

[SaltProphet]

@copilot retry

[Copilot]

@copilot retry

I'm not sure what you'd like me to retry. The implementation is complete with all requested features:

• ✅ Dark theme UI (commit 3d9a1e7)
• ✅ Hardware monitoring system
• ✅ Data visualization with charts
• ✅ All tests passing

Could you please specify:

1. What specific feature or change needs to be retried?
2. What issue or error are you experiencing?
3. What behavior would you like to see changed?

This will help me address your needs accurately.

[SaltProphet]

@copilot pr has conflicts

[Copilot]

@copilot pr has conflicts

I can see the branch was already merged with main in commit 01ec957. The current branch state shows no local conflicts.

If GitHub is still showing conflicts, this typically means the base branch (likely main) has new changes that conflict with our PR.

To resolve:

1. Identify the conflicting files - GitHub should show which files have conflicts in the PR
2. Common conflict areas in this PR:
   • .env.example - we added new config fields
   • config.py - we added new settings fields
   • dashboard.py - we added new endpoints
   • requirements.txt - we added psutil

Could you share which specific files GitHub is reporting as conflicting? That will help me create a targeted fix.