Pipeline Walkthrough

This page is the annotated tour of the two bundled entry-point scripts:

Research Workflow — main.py — one-shot calculate → train → predict → evaluate on a fixed date split.
Scenarios Workflow — scenarios.py — same pipeline executed once per scenario in a nine-scenario sweep.

Both scripts share the same ForecastModel instance and the same calculate() output; they differ in how the train → predict → evaluate legs are sequenced.

Research Workflow (`main.py`)

main.py runs a single, linear pipeline: it computes tremor over the whole year, trains on the first seven months, forecasts the next four weeks, and evaluates against the held-out eruptions.

Stage flow

┌──────────────────────────────────────────────────────────────────────┐
│                       main.py — Stage Flow                           │
└──────────────────────────────────────────────────────────────────────┘

  fm = ForecastModel(network="VG", station="OJN", location="00",
                     channel="EHZ", day_to_forecast=2,
                     n_jobs=8, verbose=True)
       │
       ▼
┌─────────────────────┐
│  fm.calculate()     │  CalculateTremor → tremor_*.csv
│                     │  source = SDS  (D:\Data\OJN)
│                     │  methods = rsam, dsar, entropy
│                     │  dates: 2025-01-01 → 2025-12-31
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│  fm.train()         │  TrainingModel → ClassifierEnsemble
│                     │    build_label  (window_step = 6 h, dtf = 2)
│                     │    extract_features (tsfresh, 5 tremor columns)
│                     │    fit          (4 classifiers × 25 seeds)
│                     │  cv = shuffle-stratified, scoring = recall
│                     │  resample = under,  n_jobs = 4, n_grids = 4
└──────────┬──────────┘
           │  cached → {station_dir}/cache/TrainingModel/{hash}.pkl
           │
           ▼
┌─────────────────────┐
│  fm.predict()       │  PredictionModel → results
│                     │    build_label (window_step = 10 min)
│                     │    extract_features (same tremor columns)
│                     │    forecast     (4 clf × 25 seeds → consensus)
│                     │  plot_threshold = 0.7,  save_seed_result = True
└──────────┬──────────┘
           │  cached → {station_dir}/cache/PredictionModel/{hash}.pkl
           │
           ▼
┌─────────────────────┐
│  fm.evaluate(       │  EvaluationModel(model="prediction")
│    model=           │    MetricsEnsemble.compute()
│    "prediction")    │    aggregate per-classifier metrics
│                     │    plot_per_seed = True
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│  fm.EvaluationModel │  ClassifierComparator
│      .compare()     │    ranking CSV + comparison plots
└─────────────────────┘

Per-stage notes

`fm.calculate(...)`

Runs once across the full year so the same tremor frame is reused by every downstream stage.
start_date is internally shifted back by day_to_forecast days (forecast_model.py:154) so the first label window has enough lead-in.
interpolate=True fills miniSEED gaps so tsfresh receives a continuous signal.
plot_daily=True emits per-day band plots under tremor/figures/ — useful for spotting station outages before training.

`fm.train(...)`

classifiers=["lite-rf", "rf", "gb", "xgb"] — four classifiers fit independently; their fitted SeedEnsembles are bundled into a single ClassifierEnsemble.
eruption_dates lists every known eruption — both the training-window eruptions and the held-out ones the forecast will be evaluated against.
scoring="recall" instructs GridSearchCV to maximise positive recall — false negatives are worse than false positives for eruption forecasting.
seeds=25 in main.py is a quick run; the default in scenarios.py is seeds=10 per scenario.
n_jobs=4, n_grids=4 — 4 outer seed workers × 4 inner CV workers. Clamped by BaseModel.validate() if the box has fewer cores.

`fm.predict(...)`

window_step=10, window_step_unit="minutes" — 144 forecasts/day. The dense grid is what makes the forecast plot smooth.
save_seed_result=True writes one CSV per seed under prediction/results/{clf-slug}/ for downstream uncertainty analysis.
use_cache=False forces a fresh forecast — flip to True to short-circuit when nothing upstream changed.
The cache identity threads in _training_cache_hash, so re-training automatically invalidates downstream predictions.

`fm.evaluate(model="prediction", plot_per_seed=True)`

Reuses the in-memory PredictionModel — no re-extraction of features, no re-fit.
Falls back to train()'s eruption_dates when called without an explicit list.
Writes per-seed JSON under evaluation/prediction/classifiers/{clf}/metrics/json/{seed:05d}.json plus aggregate metrics_summary_*.csv.
plot_per_seed=True is expensive — flip off for fast iteration.

`fm.EvaluationModel.compare()`

One MultiModelEvaluator per classifier → fed to a ClassifierComparator.
comparator.get_ranking() writes comparison/metrics/ranking_recall.csv (defaults to recall ranking — matches the training scoring).
comparator.plot_all() writes ROC overlay, metric bars, seed stability violins, and a comparison grid under evaluation/prediction/comparison/figures/.

Scenarios Workflow (`scenarios.py`)

scenarios.py keeps the same ForecastModel instance and the same tremor frame, but loops the train → predict → evaluate legs over nine scenarios that vary the training and prediction date splits. Each scenario's outputs land in its own directory so they can be compared side-by-side.

Scenarios in `scenarios.py`

#	Train window	Forecast window	Goal
1	2025-01-01 → 2025-03-31	2025-04-01 → 2025-04-30	Train on 1 eruption, forecast eruption 2
2	2025-01-01 → 2025-03-31	2025-05-01 → 2025-05-31	Train on 1 eruption, forecast eruption 3
3	2025-01-01 → 2025-03-31	2025-06-01 → 2025-06-30	Train on 1 eruption, forecast eruption 4
5	2025-01-01 → 2025-04-30	2025-05-01 → 2025-05-31	Train on 1+2, forecast eruption 3
6	2025-01-01 → 2025-05-31	2025-06-01 → 2025-06-30	Train on 1+2+3, forecast eruption 4
7	2025-01-01 → 2025-06-30	2025-07-01 → 2025-07-13	Train on 1–4, forecast eruption 5
8	2025-01-01 → 2025-07-26	2025-07-27 → 2025-08-22	Train on 1–5, forecast eruptions 6+7
9	2025-01-01 → 2025-08-22	2025-01-01 → 2025-08-22	Sanity check — train and predict over the full record

(Scenario 4 is intentionally absent in scenarios.py.)

Stage flow

┌──────────────────────────────────────────────────────────────────────────┐
│                     scenarios.py — Outer-Loop Flow                       │
└──────────────────────────────────────────────────────────────────────────┘

  fm = ForecastModel(...)                  # one ForecastModel reused
       │
       ▼
┌─────────────────────┐
│  fm.calculate()     │  one full-year tremor CSV (shared across scenarios)
└──────────┬──────────┘
           │
           ▼
   for scenario in scenarios:              # nine scenarios in scenarios.py
           │
           │  output_dir = os.path.join(
           │      root_dir, "output", fm.nslc,
           │      "scenarios", slugify(name))
           │
           ▼
   ┌───────────────────────────────────────────────────────────────────┐
   │  (scoped to scenario.output_dir)                                  │
   │                                                                   │
   │  fm.train(start, end, eruption_dates, …, output_dir=output_dir)   │
   │       │                                                           │
   │       ▼                                                           │
   │  fm.predict(start, end, …, output_dir=output_dir,                 │
   │             eruption_dates=plot_kwargs["eruption_dates"])         │
   │       │                                                           │
   │       ▼                                                           │
   │  send_telegram_notification(                                      │
   │      message=f"{name}: {description}",                            │
   │      files=[fm.PredictionModel.forecast_plot_path],               │
   │      file_caption=f"{name}: {description}")                       │
   │       │                                                           │
   │       ▼                                                           │
   │  fm.evaluate(model="prediction", plot_per_seed=True,              │
   │              output_dir=output_dir)                               │
   └───────────────────────────────────────────────────────────────────┘
           │
           ▼
   next scenario  →  loop back

Per-scenario notes

output_dir is built once per scenario from slugify(name) ("Scenario 1" → "scenario-1"). Every stage call inside the loop forwards that output_dir, so artefacts land at output/{nslc}/scenarios/{slug}/.
The outer fm.calculate(...) runs once before the loop. Its tremor frame is captured on fm.tremor_df and reused on every iteration — fm.train() reads from fm.tremor_df, not from disk.
eruption_dates is the full list of eight known eruptions on every scenario. The training window simply excludes the eruptions that haven't happened yet, and the prediction window picks them up later.
plot_kwargs["eruption_dates"] is forwarded into fm.predict(...) as **plot_kwargs, which routes them to forecast_plots so eruption-day markers are drawn on the per-scenario forecast plot.
The Telegram hook (send_telegram_notification) ships the per-scenario forecast PNG to the configured chat the moment predict() returns. The bundled send_as_document=True flag forces the file to be uploaded as a document rather than re-encoded as an image — preserves the full DPI plot.
fm.evaluate(...) runs after the notification, so by the time you see the plot in Telegram, the per-seed metrics JSON is already being computed in the background.

Resulting directory layout

output/
└── VG.OJN.00.EHZ/
    ├── tremor/                            # shared across all scenarios
    │   └── VG.OJN.00.EHZ_2025-01-01_2025-12-31.csv
    └── scenarios/
        ├── scenario-1/
        │   ├── training/
        │   ├── prediction/
        │   ├── evaluation/prediction/
        │   └── cache/
        ├── scenario-2/
        ...
        └── scenario-9/

See Output Structure for the full per-scenario tree.

Choosing Between the Workflows

	Research (`main.py`)	Scenarios (`scenarios.py`)
Number of forecasts	1	N (9 in the bundled script)
Tremor computation	once	once (reused)
Training	once	once per scenario
Output isolation	flat	one directory per scenario
Telegram notifications	per-stage via `@notify` decorator	per-scenario forecast plot via `send_telegram_notification(...)`
Use case	a single research run on a fixed split	leave-one-out style sweeps, ablation over training windows

When the goal is to publish a forecast plot, use main.py. When the goal is to compare how forecast quality degrades as the training-window shifts, use scenarios.py.

Navigation Menu

Pipeline Walkthrough

Pipeline Walkthrough

Research Workflow (`main.py`)

Stage flow

Per-stage notes

`fm.calculate(...)`

`fm.train(...)`

`fm.predict(...)`

`fm.evaluate(model="prediction", plot_per_seed=True)`

`fm.EvaluationModel.compare()`

Scenarios Workflow (`scenarios.py`)

Scenarios in `scenarios.py`

Stage flow

Per-scenario notes

Resulting directory layout

Choosing Between the Workflows

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

Pipeline Walkthrough

Pipeline Walkthrough

Research Workflow (main.py)

Stage flow

Per-stage notes

fm.calculate(...)

fm.train(...)

fm.predict(...)

fm.evaluate(model="prediction", plot_per_seed=True)

fm.EvaluationModel.compare()

Scenarios Workflow (scenarios.py)

Scenarios in scenarios.py

Stage flow

Per-scenario notes

Resulting directory layout

Choosing Between the Workflows

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

Research Workflow (`main.py`)

`fm.calculate(...)`

`fm.train(...)`

`fm.predict(...)`

`fm.evaluate(model="prediction", plot_per_seed=True)`

`fm.EvaluationModel.compare()`

Scenarios Workflow (`scenarios.py`)

Scenarios in `scenarios.py`