DataForge ⚡

All-in-one backend platform combining API Gateway, Energy Forecasting, Database Backup, and more.

Features

🔌 API Gateway

• Path-based routing
• Load balancing
• Circuit breaker
• Rate limiting
• Caching

⚡ Energy Forecasting

• Multiple ML models: Random Forest, Gradient Boosting, XGBoost, LightGBM, LSTM
• Anomaly detection
• Time series analysis
• Interactive Streamlit dashboard

💾 Database Backup

• SQLite, MySQL, PostgreSQL support
• Binary and SQL dump backups
• Compression support

🔐 Authentication

• API Key authentication
• JWT token support
• Role-based access control

🐳 Docker Support

• Production-ready Dockerfile
• docker-compose for local development
• Redis integration

Installation

# Core
pip install dataforge

# With ML models
pip install dataforge[ml]

# With dashboard
pip install dataforge[dashboard]

# Full installation
pip install dataforge[ml,dashboard]

Quick Start

CLI

# Energy forecasting
dataforge energy train data/energy.csv --model-type rf
dataforge energy forecast data/energy.csv --steps 24
dataforge energy analyze data/energy.csv

# Database backup
dataforge backup create sqlite mydb.db
dataforge backup list

# Authentication
dataforge auth create-key myapp --roles admin
dataforge auth create-token user --roles user

API Server

dataforge serve api

Dashboard

dataforge serve dashboard

Docker

docker-compose up -d

Available Commands

Command	Description
dataforge energy train	Train forecasting model
dataforge energy forecast	Generate forecasts
dataforge energy analyze	Analyze data
dataforge backup create	Create backup
dataforge backup restore	Restore backup
dataforge auth create-key	Create API key
dataforge auth create-token	Create JWT token
dataforge serve api	Start API server
dataforge serve dashboard	Start dashboard

ML Models

Model	Description
rf	Random Forest
gb	Gradient Boosting
xgb	XGBoost
lgbm	LightGBM
lstm	LSTM Neural Network

API Endpoints

Energy

• POST /energy/train - Train model
• POST /energy/forecast - Generate forecast
• POST /energy/anomaly - Detect anomalies

Backup

• POST /backup/create - Create backup
• POST /backup/restore - Restore backup
• GET /backup/list - List backups

Architecture

DataForge/
├── dataforge/
│   ├── api/          # API Gateway
│   ├── energy/       # Energy forecasting
│   ├── database/     # Database backup
│   ├── auth/         # Authentication
│   └── common/       # Shared utilities
├── tests/
├── data/
├── Dockerfile
└── docker-compose.yml

Docker Services

services:
  api:        # FastAPI server (port 8000)
  dashboard:  # Streamlit dashboard (port 8501)
  redis:      # Redis cache

Environment Variables

• DATAFORGE_HOST - Server host
• DATAFORGE_PORT - Server port
• DATAFORGE_REDIS_URL - Redis URL
• DATAFORGE_BACKUP_DIR - Backup directory

License

MIT License