Prediction Workflow

The prediction stage runs forecast inference on a future (or held-out) time window using a previously trained ClassifierEnsemble. No labels are required at this stage - predictions are produced over an evenly-spaced window grid.

Driver: PredictionModel (src/eruption_forecast/model/prediction_model.py). Wrapped by ForecastModel.predict(...).

Internal Pipeline

                ┌──────────────────────────────────────────────┐
                │             PredictionModel                  │
                │   inherits BaseModel                         │
                │   self.ClassifierEnsemble loaded eagerly     │
                └──────────────┬───────────────────────────────┘
                               │
                  ┌────────────┼─────────────┐
                  ▼            ▼             ▼
        ┌─────────────┐  ┌─────────────┐  ┌─────────────┐
        │ build_label │→ │ extract_    │→ │  forecast   │
        │             │  │ features    │  │             │
        └─────────────┘  └─────────────┘  └─────────────┘
            unlabelled       prediction-      ClassifierEnsemble
            window grid      mode tsfresh     .predict_with_uncertainty
            (id + datetime)  (no relevance    → per-classifier +
                              filtering)        consensus probabilities

ForecastModel.predict(...) chains all three internally.

Trained Model Sources

PredictionModel(model=...) accepts five forms via ClassifierEnsemble.from_any():

Source	Example
`ClassifierEnsemble` object	`fm.TrainingModel.ClassifierEnsemble`
`SeedEnsemble` object	a single-classifier ensemble (auto-wrapped)
Path to `ClassifierEnsemble*.pkl`	`output/.../training/classifiers/ClassifierEnsemble_StratifiedShuffleSplit.pkl`
Path to `ClassifierEnsemble*.json`	the JSON registry written next to the `.pkl`
Path to a `SeedEnsemble_*.pkl`	bundle for a single classifier
Path to a trained-model registry `.json` (new) or `.csv` (legacy)	`trained-model__RandomForestClassifier_...json`

When called via fm.predict(...), the in-memory fm.ClassifierEnsemble is passed directly - no disk round-trip.

Build Forecast Grid (`build_label`)

Despite the name, prediction labels are placeholders - every is_erupted value is 0. The role of build_label() here is to lay out the window grid the model will score against.

Param	Type	Notes
`window_step`	`int`	Stride between consecutive forecast windows
`window_step_unit`	`"minutes"` \| `"hours"`	A 10-minute step produces 144 forecasts/day

The grid is cached as {prediction_dir}/features/features-label_{basename}_step-{N}-{unit}.csv. On a re-run the cached grid is loaded unless overwrite=True.

Extract Features (`extract_features`)

Runs the same TremorMatrixBuilder → FeaturesBuilder chain as training, but in prediction mode - no tsfresh relevance filtering, because there are no labels to test relevance against.

select_tremor_columns, save_tremor_matrix_per_method, and exclude_features are mirrored from the training call so that the prediction feature columns line up with the trained model's input schema. ForecastModel.predict(...) forwards self.select_tremor_columns, self.save_tremor_matrix_per_method, and self.exclude_features from the upstream train() call automatically.

PredictionModel.extract_features(...) also accepts a select_features kwarg (a curated list[str] or a path to a top_{N}_features.csv / top_features.csv / significant_features.csv). When supplied, it is resolved via load_select_features(..., number_of_features=0) and forwarded as the tsfresh kind_to_fc_parameters — only the named features are computed per tremor column instead of the full ComprehensiveFCParameters set. ForecastModel.predict(use_features_from=...) picks the value for you; see the next section.

Feature Scoping via `use_features_from`

ForecastModel.predict(...) exposes a single knob, use_features_from: Literal["all", "files", "training"] = "all", that switches between three ways of scoping features for the forecast window.

Mode	What `predict()` runs	When to reach for it
`"all"` (default)	`build_label → extract_features(select_features=None)` — every tsfresh feature is computed per tremor column	Cold start; no prior features on disk; want to keep the forecast independent of which features the training run happened to pick
`"files"`	`load_features(features_matrix_path, label_features_csv)` — tsfresh is skipped entirely and a pre-built matrix is loaded	Replaying a forecast from a shared feature matrix; want to swap classifiers against a fixed feature snapshot; forecasting from just two files + a `ClassifierEnsemble.pkl` without pointing at raw tremor data
`"training"`	`build_label → extract_features(select_features=<union of training-time features>)` — tsfresh is narrowed to the features any seed picked during `train()`	Realistic operational default: the narrower matrix is smaller and faster while still covering every feature the ensemble was actually trained on

Under the hood, predict() derives select_features and dispatches accordingly:

if use_features_from == "files":
    #  Both paths must be supplied and must exist on disk.
    #  use_cache is forced to False because load_features()
    #  bypasses the extract-features kwargs that populate the
    #  prediction cache identity.
    select_features = None
    prediction_model.load_features(features_matrix_path, label_features_csv, ...)

elif use_features_from == "training":
    #  Auto-narrow tsfresh to the union of features any seed picked
    #  during train(). Falls back to None (extract everything) when
    #  the trained ensemble has no features_selected_df — e.g. a
    #  legacy cache with no feature-selection record.
    select_features = (
        self.TrainingModel.features_selected_df.index.tolist()
        if not self.TrainingModel.features_selected_df.empty
        else None
    )
    prediction_model.build_label(...).extract_features(select_features=select_features, ...)

else:  # "all"
    select_features = None
    prediction_model.build_label(...).extract_features(select_features=None, ...)

Validation and failure modes

use_features_from="files" is a hard-fail on misconfiguration — no silent fall-back to "all".

Condition	Raises
`features_matrix_path` or `label_features_csv` is `None`	`ValueError("use_features_from='files' requires both features_matrix_path and label_features_csv to be provided.")`
Either supplied path does not exist on disk	`FileNotFoundError(f"features_matrix_path does not exist: {...}")` (or `label_features_csv`)
Only one of the two paths is supplied (independent of mode)	`ValueError("features_matrix_path and label_features_csv must be provided together.")`

Under use_features_from="files", use_cache is forced to False because load_features() does not populate the extract-features kwargs the prediction cache identity depends on — the cache lookup would be meaningless.

Cache identity implications

"all" and "training" both flow through extract_features(...) and therefore fold select_features into the PredictionModel.build_identity(...) extract-features kwargs. Switching between the two invalidates the prediction cache pickle, which is intentional — the two produce measurably different feature matrices.
"files" disables the cache outright, so its results are always freshly computed against the supplied matrix.

Config round-trip

use_features_from is captured on ForecastPredictConfig, so a fm.save_config() → ForecastModel.from_config(...) → fm.run() cycle preserves the mode. features_matrix_path and label_features_csv round-trip as absolute path strings; leave them null in the YAML unless the mode is "files".

Choosing between the modes

# Cold start — no prior features on disk.
fm.predict(..., use_features_from="all")

# Preferred once a training run exists — narrower matrix, same feature set.
fm.predict(..., use_features_from="training")

# Replay a forecast against a pre-built matrix.
fm.predict(
    ...,
    use_features_from="files",
    features_matrix_path="output/.../prediction/features/features-matrix_....parquet",
    label_features_csv="output/.../prediction/features/features-label_....csv",
)

Forecast (`forecast`)

results = ClassifierEnsemble.predict_with_uncertainty(
    X=features_df,
    save=save_seed_result,             # write per-seed CSV under result_dir/{clf}/
    output_dir=result_dir,
    overwrite=overwrite,
)

The DataFrame returned by forecast() (also stored on self.results and fm.results) has one row per forecast window and the following columns:

Column suffix	Meaning
`{clf}_eruption_probability`	Mean P(eruption) across the classifier's seeds
`{clf}_uncertainty`	Std-dev across the classifier's seeds
`{clf}_confidence`	`1 - normalised_uncertainty`
`{clf}_prediction`	Binary prediction at the seed-averaged threshold
`consensus_eruption_probability`	Mean of `{clf}_eruption_probability` across classifiers
`consensus_uncertainty`	Pooled std across classifiers + seeds
`consensus_confidence`	`1 - normalised consensus uncertainty`
`consensus_prediction`	Binary prediction on the consensus mean

The result CSV is written at {station_dir}/forecast-results_{basename}.csv.

Plotting

prediction/figures/forecast_{basename}.png       # always
prediction/figures/forecast_{basename}.pdf       # when plot_pdf=True (default)

`forecast(...)` param	Default	Effect
`save_seed_result`	`True`	Per-seed CSVs under `prediction/results/{clf}/`
`plot_threshold`	`0.5`	Horizontal threshold line on the forecast plot
`plot_title`	`None`	Optional title
`plot_pdf`	`True`	Also save a vector PDF
`**plot_kwargs`	-	Forwarded to `eruption_forecast.plots.plot_forecast` - e.g. `eruption_dates=[...]` to render eruption markers

Cache

PredictionModel inherits the cache layer from BaseModel. The cache identity includes:

NSLC (constructor param, threaded by ForecastModel.predict)
tremor DataFrame fingerprint
training_hash (constructor param) - the cache hash of the upstream TrainingModel
start_date, end_date, window_size
build_label kwargs (window_step, window_step_unit)
extract_features kwargs (select_tremor_columns, save_tremor_matrix_per_method, exclude_features)

Threading training_hash means re-training automatically invalidates the prediction cache. forecast() calls self.save(self.build_identity()); the pickle lands at {station_dir}/prediction/{hash}.PredictionModel.pkl + {hash}.PredictionModel.params.json.

Outputs

{station_dir}/
├── prediction/
│   ├── features/
│   │   ├── features-label_{basename}_step-{N}-{unit}.csv  # forecast grid
│   │   └── features-matrix_*.parquet                       # tsfresh matrix (Snappy Parquet)
│   ├── results/{clf-slug}/{seed:05d}.csv                   # per-seed probability (save_seed_result=True)
│   └── figures/forecast_{basename}.{png,pdf}               # forecast plot
├── forecast-results_{basename}.csv             # top-level results dump
└── prediction/{hash}.PredictionModel.pkl       # content-addressable cache pickle (+ .params.json sidecar)

fm.PredictionModel.forecast_plot_path exposes the path to the rendered plot - used by scenarios.py to attach the figure to a Telegram notification.

Standalone Use

from eruption_forecast import PredictionModel

pm = (
    PredictionModel(
        model="output/VG.OJN.00.EHZ/training/classifiers/ClassifierEnsemble_StratifiedShuffleSplit.pkl",
        tremor_data="output/VG.OJN.00.EHZ/tremor/VG.OJN.00.EHZ_2025-01-01_2025-12-31.csv",
        start_date="2025-07-27",
        end_date="2025-08-22",
        window_size=2,                 # must match the trained model's window_size
        output_dir="output/VG.OJN.00.EHZ",
        n_jobs=4,
    )
    .build_label(window_step=10, window_step_unit="minutes")
    .extract_features(
        select_tremor_columns=["rsam_f2", "rsam_f3", "rsam_f4", "dsar_f3-f4", "entropy"],
    )
)

df_forecast = pm.forecast(
    plot_threshold=0.7,
    plot_pdf=True,
    eruption_dates=["2025-08-02", "2025-08-18"],   # → forecast_plots plot kwargs
)

pm.save()                       # → {output_dir}/PredictionModel_{basename}.pkl
print(pm.forecast_plot_path)    # path to the saved plot
print(df_forecast.head())       # 10-minute resolution forecast

Reload a saved `.pkl`

pm = PredictionModel.load("output/VG.OJN.00.EHZ/PredictionModel_2025-07-27_2025-08-22.pkl")
df = pm.results

The reloaded pm.results is the same DataFrame returned by forecast() - no re-inference needed for downstream analysis.

Reuse a feature matrix from a prior run

PredictionModel.load_features(...) skips tsfresh entirely and loads a previously written features-matrix_*.parquet + features-label_*.csv. Use it when the windowing and tremor data have not changed — for example replaying forecast() with a different trained ensemble against an existing feature matrix, or forecasting from just two files (a ClassifierEnsemble.pkl plus a pre-computed features matrix) without ever pointing at raw tremor data:

from eruption_forecast import PredictionModel

pm = PredictionModel(
    model="output/VG.OJN.00.EHZ/training/classifiers/ClassifierEnsemble_StratifiedShuffleSplit.pkl",
    tremor_data="output/VG.OJN.00.EHZ/tremor/VG.OJN.00.EHZ_2025-01-01_2025-12-31.csv",
    start_date="2025-07-27",
    end_date="2025-08-22",
    window_size=2,                 # must match the trained model's window_size
    output_dir="output/VG.OJN.00.EHZ",
    n_jobs=4,
).load_features(
    features_matrix_path="output/VG.OJN.00.EHZ/prediction/features/features-matrix_2025-07-27_2025-08-22_step-10-minutes.parquet",
    label_features_csv="output/VG.OJN.00.EHZ/prediction/features/features-label_2025-07-27_2025-08-22_step-10-minutes.csv",
    window_step=10,
    window_step_unit="minutes",
)

df_forecast = pm.forecast(plot_threshold=0.7, plot_pdf=True)

load_features() drops in as a replacement for the build_label() → extract_features() prefix before forecast(). Both paths are required — no auto-resolve fallback — so intent is always explicit and a stale features-*_* artefact in self.features_dir cannot be picked up silently. The loader validates that the label CSV's sampling matches the caller-supplied window_step / window_step_unit and that its datetime span covers the configured [start_date, end_date] forecast range. ForecastModel.predict(use_features_from="files", features_matrix_path=..., label_features_csv=...) threads the same call from the orchestrator — see Feature Scoping via use_features_from above for the gate; the mode kwarg is required (without it the two paths are silently ignored under the "all" default).

Persist the prediction config

pm.save_config()   # → {prediction_dir}/prediction.config.yaml

forecast() already auto-calls save_config() at the end, so a standalone prediction run always leaves a YAML snapshot at {output_dir}/prediction/prediction.config.yaml. The captured model and tremor_data fields preserve the path strings the user supplied (or null when a live ensemble / pre-loaded DataFrame was passed). See Configuration.

Navigation Menu

Prediction Workflow

Prediction Workflow

Internal Pipeline

Trained Model Sources

Build Forecast Grid (`build_label`)

Extract Features (`extract_features`)

Feature Scoping via `use_features_from`

Validation and failure modes

Cache identity implications

Config round-trip

Choosing between the modes

Forecast (`forecast`)

Plotting

Cache

Outputs

Standalone Use

Reload a saved `.pkl`

Reuse a feature matrix from a prior run

Persist the prediction config

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

Prediction Workflow

Prediction Workflow

Internal Pipeline

Trained Model Sources

Build Forecast Grid (build_label)

Extract Features (extract_features)

Feature Scoping via use_features_from

Validation and failure modes

Cache identity implications

Config round-trip

Choosing between the modes

Forecast (forecast)

Plotting

Cache

Outputs

Standalone Use

Reload a saved .pkl

Reuse a feature matrix from a prior run

Persist the prediction config

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

Build Forecast Grid (`build_label`)

Extract Features (`extract_features`)

Feature Scoping via `use_features_from`

Forecast (`forecast`)

Reload a saved `.pkl`