A cleaned, deduplicated, quality-gated open dataset of global football match data for Both-Teams-To-Score (BTTS) and goals modelling. Sourced from API-Football and football-data.co.uk.
This repository holds the pipeline code, docs, metadata and ~1k-row samples. The full data is published on Hugging Face:
eatpizzanot/soccer-dataset(parquet,load_dataset()-ready).
| Metric | Value |
|---|---|
| Fixtures | 673,966 (644,901 played) |
| Leagues | 271 |
| Teams | 11,104 |
| Players | 182,125 |
| Date range | 2008-06-07 - 2027-06-06 |
| BTTS base rate | 0.5063 |
| Fixtures with xG | 128,746 |
| Fixtures with odds | 186,813 |
- League history is uneven. Major leagues carry ~10+ years; some newly-added
Cloudbet-covered leagues have shorter history. Per-league coverage is in
league_catalogue(history_status= full / recent_only / partial) — check it before assuming a league is complete. - xG is a coarse, provider-supplied estimate — not a real per-shot model. API-Football's
xG is a deterministic shots-by-zone formula (empirically
xg ≈ 0.115·shots_inside_box + 0.035·shots_outside_box + 0.648·penalties, R²≈1.0); it carries no shot-quality signal beyond zone counts and correlates only ~0.4 with actual goals. It is nulled entirely for league-seasons the provider does not cover (seexg_covered); never treat missing xG as0.
- Reconciled two divergent source snapshots (CSV + Parquet) by a consistent internal id.
- Deduplicated cross-source duplicate matches on a canonical key (league, UTC day, home, away).
- xG fake-zero fix: xG stored as
0for leagues API-Football does not cover for xG has been set toNULL(detected per league-season). Never treat missing xG as0. - Entity integrity: cross-source duplicate matches merged; 25 upstream "team-plays-itself" fixtures (from a provider club-merger over-merge) dropped with an audit trail.
- Leakage guard: post-match facts carry a
known_attimestamp. - 12-dimension QA gate (
QUALITY_REPORT.md, 50+ blocking checks) must pass before publishing.
See docs/data_dictionary.md for every column. Core tables:
fixtures, match_stats, odds, fixture_lineups, teams, players, leagues,
fixture_players, fixture_players_stats_flat, a league_catalogue
(dataset x API-Football x Cloudbet coverage, incl. per-league history_status), and
xg_training (team-match shot-aggregate → real-xG subset for lite-xG modelling).
from datasets import load_dataset
fixtures = load_dataset("eatpizzanot/soccer-dataset", data_files="fixtures.parquet")["train"]import pandas as pd
fx = pd.read_parquet("https://huggingface.co/datasets/eatpizzanot/soccer-dataset/resolve/main/fixtures.parquet")
print(fx.query("is_played").eval("goals_home>0 and goals_away>0").mean()) # BTTS rateThe dataset is regenerated by a single, idempotent, re-runnable script (not a scheduled
service): python scripts/restore.py. See docs/PIPELINE.md.
Data is compiled from publicly available sources for research/educational use, released under CC-BY-4.0. Please cite API-Football and football-data.co.uk.