Phase 4: Add statistical structural-break and change-point models - #9
Conversation
- Add detectors.py with a shared BreakDetector interface and three detectors: CusumDetector (single dominant mean shift), RollingZScoreDetector (local deviations/transitions), and PeltDetector (PELT via ruptures, multiple breaks) - Add synthetic.py: reproducible series with known break points (mean shift, variance shift, multiple breaks) - Extend evaluation.py with extract_break_points and windowed point-based precision/recall/F1 metrics - Add visualization.py: optional matplotlib plotting helper (lazy import) - Add scripts/compare_methods.py: runs detectors + ML baseline on synthetic data, scores them, and writes outputs/method_comparison.csv (+ optional figure) - Add ruptures and matplotlib to requirements - Add tests for detectors, evaluation utilities, synthetic data, and plotting - Update README with a methods table, comparison workflow, and synthetic results
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a1237c51a7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| scipy>=1.7 | ||
| joblib>=1.1 | ||
| # Change-point detection and visualization. | ||
| ruptures>=1.1 |
There was a problem hiding this comment.
Adding ruptures only to requirements.txt leaves the [project].dependencies list in pyproject.toml unchanged. Users who install the project through pip install ., an editable install without first installing the requirements file, or a built package will not receive ruptures, so the newly advertised PeltDetector.detect() raises ImportError. Add this core detector dependency to the package metadata as well.
Useful? React with 👍 / 👎.
| change_point = int(np.argmax(np.abs(cumulative))) | ||
| if score[change_point] >= self.threshold: | ||
| flags[change_point] = 1 |
There was a problem hiding this comment.
For CUSUM, the extremum at index k represents the split immediately after observation k, but this code flags k itself. For example, the noiseless series [0] * 60 + [3] * 60 flags row 59 even though this package's synthetic-data convention and PELT output define the break as the first new-regime row, index 60. This systematically shifts CUSUM detections and makes exact-location evaluation and plots inconsistent across detectors.
Useful? React with 👍 / 👎.
| rolling = series.rolling(self.window, min_periods=2) | ||
| mean = rolling.mean() | ||
| std = rolling.std(ddof=0) |
There was a problem hiding this comment.
Because the rolling mean and standard deviation include the sample being scored, the absolute z-score is bounded by sqrt(window - 1). Consequently, configurations such as RollingZScoreDetector(window=5) with the default threshold of 3 can never flag any observation, regardless of how extreme it is. Shift the rolling statistics before comparing the current sample, or reject thresholds that make detection impossible.
Useful? React with 👍 / 👎.
| predicted = extract_break_points(result["has_structural_break"].to_numpy()) | ||
| for i, point in enumerate(predicted): | ||
| ax.axvline( | ||
| timestamps.iloc[point], |
There was a problem hiding this comment.
When the accepted input frame is not already timestamp-sorted, BreakDetector.detect() sorts it before producing result, but this loop interprets result-array indices against timestamps from the original unsorted df. Calling the intended plot_detection(df, detector.detect(df)) flow can therefore draw predicted break lines at unrelated timestamps. Use the timestamps from the aligned detector result, and sort the plotted series consistently.
Useful? React with 👍 / 👎.
2 participants
Summary
Adds domain-appropriate change-point detectors alongside the ML baseline, turning the project into a credible structural-break research repository. Closes #4.
detectors.py): every detector implementsBreakDetectorand exposesdetect(df) -> DataFrame[timestamp, break_score, has_structural_break, method]. Input is validated and sorted bytimestamp; row count is preserved.CusumDetector— cumulative-sum mean-shift detector locating the single dominant change point (argmax|S_k|) with a configurable significance threshold.RollingZScoreDetector— transparent local-deviation detector flagging rows beyond a configurable rolling z-score threshold.PeltDetector— PELT segmentation viaruptures, finding one or more mean shifts with a configurable model/penalty (BIC-style default).synthetic.py): reproduciblemake_mean_shift,make_variance_shift, andmake_multiple_breaksgenerators returning(df, break_points).evaluation.py):extract_break_points(rising-edge → break indices) andpoint_based_metrics(windowed precision/recall/F1 with greedy one-to-one matching).visualization.py): optionalplot_detectionhelper (matplotlib imported lazily).scripts/compare_methods.py): runs all detectors + the supervised baseline on a chosen synthetic dataset, scores them, and writesoutputs/method_comparison.csv(git-ignored) plus an optional figure.rupturesandmatplotlibto requirements; updates the README with a methods table, the comparison command, and a clearly-labelled synthetic results table.Documented assumptions
has_structural_breakmarks located break row(s); contiguous flags collapse to one detected break for point-based scoring. The row-level vs. point-level conversion is documented in the module docstring and each class.Motivation
A Random Forest on lag/rolling features is a generic classifier, not a structural-break method. CUSUM, rolling z-score, and PELT are purpose-built for change-point/regime-shift detection, and comparing them on data with a known ground truth is what makes this a structural-break research project rather than a generic ML baseline.
Validation
Detector tests cover output columns, row-count preservation, binary flags, clear errors for missing columns, small-series handling, and at-least-one synthetic mean-shift detected near the true break. The generated comparison CSV and figures stay git-ignored.
Follow-up Work
docs/portfolio-polish):docs/methodology.md,docs/data.md,CONTRIBUTING.md, a curated committed example figure, and README portfolio polish.https://claude.ai/code/session_01Ddjy7PGk1UdXXcn95NyrGt
Generated by Claude Code