Peek: Is Your Model Peeking at the Future?

A Python library that audits time-series ML pipelines for look-ahead bias and data leakage — and tells you exactly where the leak is, not just that your score looks suspicious.

---

The problem

Your model scores 0.95 offline. It falls apart in production. Somewhere, a feature saw data it shouldn't have — a centered rolling window, a normalization fit on the whole dataset, a target-encoded column, a CV split with the wrong dates on the wrong side. This is look-ahead bias / data leakage, and it is one of the most common, most expensive mistakes in applied ML — and almost universally under-tooled. sklearn.TimeSeriesSplit only splits your data; it does not check whether your features or your evaluation harness are honest.

Peek is the check that's missing. Point it at a dataframe, a feature function, a CV split, or a full pipeline, and it screams — with proof — when something isn't causal.

$ peek demo

🔍 peek report ──────────────────────────
✗ CRITICAL  feature 'future_return_leak' is a near-copy of the target shifted 0 step(s)
            corr(feature['future_return_leak'], target.shift(-0)) = 1.0000
✗ CRITICAL  feature 'centered_ma_5' changes value when future rows are removed
            at row 49, value computed on the full series (106.998...) differs
            from the value computed with only data up to that row. This
            feature is not causal — it used future information.

Verdict: LEAKING — 2 critical issue(s) found. This score is fiction until fixed.

Install

pip install peek-audit
peek demo

The console command is peek; the PyPI distribution is named peek-audit (the peek name was already taken).

Quickstart

import peek

report = peek.audit(
    df,                       # your full time-ordered dataframe
    time_col="date",
    target="y",
    feature_fn=build_features,          # optional -> enables the causality check
    splits=my_cv_splits,                # optional -> enables the split check
    pipeline=model, cv=tscv, scorer=r2_score,  # optional -> enables the shuffle check
)

print(report)
report.has_leak      # bool
report.verdict        # "LEAKING" | "SUSPICIOUS" | "CLEAN"
report.to_dict()       # for CI / JSON output

What it checks

Check	Needs	Catches
target_leak	just df	A feature that's a near-exact copy of the (possibly shifted) target — always runs. Non-numeric columns it can't check are flagged, not silently skipped.
causality (flagship)	feature_fn	Recomputes your features on a truncated series and compares against the full computation. If a value changes, the feature saw the future — catches centered rolling windows, full-series normalization, whole-dataset target encoding, anything. Detects and warns (rather than false-flags) on non-deterministic or length-changing feature_fns.
split	splits or splitter	Train/test temporal overlap, training rows dated after the test window starts, missing purge/embargo gaps (works for datetime or integer time columns).
shuffle	pipeline + cv + scorer	Permutation sanity check: refits your exact pipeline+CV on randomly shuffled labels. If the real score isn't clearly better than the shuffled-label null, the harness — not just a feature — may be leaking.

Honesty note: the target_leak check is definitive but narrow (it only catches direct future-copies, and only on numeric columns). The causality and shuffle checks are the rigorous ones — they don't just look for suspicious correlations, they prove (or disprove) causality by truncation and permutation. No check here proves the absence of leakage; each one proves the presence of a specific, well-defined failure mode. A CLEAN verdict names exactly which checks ran, so you always know how much ground was actually covered — run all four you can.

CLI

peek demo                                          # instant leaky-vs-clean walkthrough
peek audit data.csv --time date --target y         # audit a CSV (target_leak check)
peek audit data.csv --time date --target y --json  # JSON output, for CI

Why this exists

This project started inside AgentQuant, an autonomous LLM research agent that proposed and backtested trading strategies. While building its rigorous walk-forward validation, we kept catching our own pipeline quietly cheating — a rolling feature with a centered window, a warmup period that was one bar short. We built a warmup enforcer and a lookback guard to stop lying to ourselves. Then we realized: this problem isn't specific to finance. Any time-series ML pipeline can leak the same way. Peek is that guard, generalized and pulled out into its own library.

Project layout

peek/                    # the library
├── audit.py             # audit() orchestrator
├── report.py            # Finding / Severity / AuditReport
├── datasets.py          # synthetic leaky/clean datasets used by `peek demo`
├── cli.py               # `peek demo`, `peek audit`
└── checks/
    ├── target_leak.py   # future-copy-of-target detector
    ├── causality.py     # truncation-based causality proof (flagship)
    ├── split.py         # train/test temporal overlap + embargo
    └── shuffle.py       # permutation sanity test on a full pipeline

tests/                   # test suite

Testing

pip install -e ".[dev]"
pytest tests/ -v

24 tests passing — target-leak, causality, split, shuffle, and report/verdict logic.

License

MIT