MLOps Batch Signal Pipeline

A minimal MLOps-style batch job that loads 10k rows of BTC/USD OHLCV data, computes a 5-period rolling mean on close, generates a binary trading signal, and outputs structured metrics — fully reproducible and Dockerized.

---

Project Structure

mlops-task/
├── run.py                  # CLI entry point
├── src/
│   ├── __init__.py
│   ├── config.py           # YAML loading + validation
│   ├── pipeline.py         # Rolling mean + signal computation
│   └── metrics.py          # Metrics building + JSON writing
├── tests/
│   └── test_pipeline.py    # Unit tests
├── data.csv                # 10k-row OHLCV dataset (BTC/USD 1-min, 2024)
├── config.yaml             # Pipeline configuration
├── requirements.txt
├── Dockerfile
├── .dockerignore
├── Makefile
└── README.md

---

Configuration (config.yaml)

seed: 42
window: 5
version: "v1"

---

Signal Logic

• Rolling mean on close using window=5, min_periods=5
• First 4 rows produce NaN rolling_mean → excluded from signal
• signal = 1 if close > rolling_mean, else 0
• signal_rate = mean(signal) over valid rows only
• rows_processed = total rows in CSV (all 10000)

---

Local Run

# 1. Install dependencies
pip install -r requirements.txt

# 2. Run the pipeline
python run.py --input data.csv --config config.yaml --output metrics.json --log-file run.log

# 3. Run tests
pytest tests/ -v

---

Docker

# Build
docker build -t mlops-task .

# Run
docker run --rm mlops-task

Using Make

make install     # pip install
make run         # local python run
make test        # pytest
make build       # docker build
make docker-run  # docker run --rm

---

Example metrics.json (success)

{
  "version": "v1",
  "rows_processed": 10000,
  "metric": "signal_rate",
  "value": 0.4991,
  "latency_ms": 134,
  "seed": 42,
  "status": "success"
}

Example metrics.json (error)

{
  "version": "v1",
  "status": "error",
  "error_message": "Required column 'close' not found. Columns present: ['open', 'high', 'low', 'volume']"
}

Metrics JSON is always written — on both success and failure.

---

Validation Handled

Case	Behaviour
Missing input file	FileNotFoundError → error JSON
Missing config file	FileNotFoundError → error JSON
Outer-quoted CSV rows	Auto-stripped and re-parsed
Empty CSV	ValueError → error JSON
Missing close column	ValueError → error JSON
Missing config field	KeyError → error JSON
Wrong config field type	TypeError → error JSON
window < 1	ValueError → error JSON