eu_power_forecaster

European electricity day-ahead price forecasting and prompt curve trading pipeline for the Germany/Luxembourg (DE/LU) market.

Highlights

• Hybrid structural + ML workflow using residual load and outage fundamentals with ML residual learning.
• Probabilistic forecasts (q10, q50, q90) for trading risk-aware decisions.
• Explainability via SHAP top feature attribution after each retrain.
• Strict expanding walk-forward validation with Diebold-Mariano significance testing.
• Prompt curve translation with empirical risk premium adjustment before trade signal generation.
• Two Anthropic integrations with full JSON audit logs and hard fallback paths.

Setup

1. Create and activate a virtual environment.
2. Install dependencies:

pip install -r requirements.txt

3. Copy environment template:

cp .env.example .env

4. Set keys in .env when running non-demo mode.

Run Commands

• Full demo, no API keys required:

python pipeline.py --demo

• Live API mode:

python pipeline.py --start-date 2021-01-01 --end-date 2026-01-31

• Live mode without ENTSO-E token (Energy-Charts provider):

python pipeline.py --provider energy-charts --start-date 2024-01-01 --end-date 2025-12-31

• Optional forward prices for signal generation:

python pipeline.py --demo --forward-prices '{"next_week_baseload":95,"next_week_peakload":110,"next_month_baseload":90,"next_month_peakload":105}'

Data Sources

Source	Endpoint	Fields Used	Notes
ENTSO-E	https://web-api.tp.entsoe.eu/api	A44, A65, A69 (B16/B18/B19), A80, optional A61	Monthly chunking, retries with backoff, UTC handling
Energy-Charts	https://api.energy-charts.info	Day-ahead price, day-ahead load/solar/wind forecasts, generation-based outage proxies	No API key; use --provider energy-charts
Open-Meteo Archive	https://archive-api.open-meteo.com/v1/archive	temperature_2m, wind_speed_10m, wind_speed_100m, shortwave_radiation, cloud_cover	Historical weather
Open-Meteo Historical Forecast	https://historical-forecast-api.open-meteo.com/v1/forecast	Same weather vars (vintages)	Used for anti-leakage/revision proxies

Outputs

Figures (outputs/figures/)

• price_history.png
• walk_forward_predictions.png
• shap_summary.png
• prompt_curve_fair_value.png
• model_comparison.png

Reports (outputs/reports/)

File	Description
submission.csv	Delivery-hour forecast output (forecast_price_eur_mwh, q10/q50/q90)
walk_forward_predictions.csv	OOS predictions from strict expanding walk-forward
future_predictions.csv	Forward horizon hourly predictions for prompt products
metrics.csv	MAE, RMSE, MASE, spike MAE, negative-price MAE, pinball
diebold_mariano.csv	Pairwise DM statistics and p-values
shap_top_features.csv	SHAP top features by retrain window
risk_premium_timeseries.csv	Rolling risk premium estimates
prompt_curve_signals.csv	Product-level fair value and signal decisions
qa_execution.json	LLM/static QA execution result
commentary.json	LLM/fallback daily commentary audit
daily_commentary.txt	Final generated commentary text

Logs (logs/)

• llm_qa_*.json: prompt hash, raw response, parse status, rules, execution results.
• commentary_*.json: prompt hash, payload, raw response, parse status, final commentary.

Prompt Curve Translation Method

Core relationship:

Forward price = E[spot over delivery] + risk premium

Implemented process:

1. Build product-level forecast moments from hourly ensemble + quantiles:
   • next-week baseload/peakload
   • next-month baseload/peakload
2. Estimate empirical risk premium as rolling 90-day mean of (realised spot - lagged model forecast) by season and hour bucket.
3. Compute adjusted fair value:
   • adjusted_fair_value = forecast_mean + risk_premium_estimate
4. If forward price is provided, compute trading z-score:
   • z = (adjusted_fair_value - forward_price) / forecast_std
5. Map z-score to signal (Long/Short/Flat) and capped position size.
6. Force invalidation to Flat if uncertainty, spike probability, wind revision shock, or insufficient hourly coverage triggers are hit.

Notes

• DST correctness is preserved by generating hourly boundaries from Europe/Berlin local time and converting to UTC; 23h/25h days are valid.
• Demo mode injects 2022 crisis spikes, negative-price tails, realistic seasonality, and residual-load correlation.
• API keys are environment-driven only; never hardcoded.