Evaluation Pipeline for Gaussian Process Emulators with High-dimensional Dataset

Gaussian Processes (GPs) are suffering from the "curse of the dimensionality". As input or output dimension grows up, the computation becomes intractable. This project aims to explore the state-of-the-art research of dimensionality reduction in Gaussian Process emulation. In this repository, a workflow using Nextflow (a orchestrated workflow management framework) is constructed to facilitate benchmarking different Gaussian Process models on high-dimensional input/output problems with minimal efforts.

Table of Contents

• Workflow
• Datasets
• Prerequisites
• Quick Start
• Folder Structure
• Advanced Usage
  • Parameter yaml file
  • SLURM
• Implementation Details
• Advantage of using Nextflow
  • Language-Agnostic Design
  • CI-managed Reproducible Environments
• Comparison between Nextflow and Snakemake
• Reference

Workflow

The pipeline follows a 4-step workflow:

1. Data Setup: Generate synthetic data or fetch and process real-world data using specialized modules

2. Preprocessing: Standardize, split, and save data in HDF5 format (language-agnostic)

3. Model Evaluation: Train and evaluate GP models in parallel:

   High-dimensional input models:

   • ExactGP (Python/GPyTorch)
   • DKL (Python/GPyTorch)
   • RGaSP (R/RobustGaSP)
   • PCA-RGaSP (R/RobustGaSP + PCA)

   High-dimensional output models:

   • PPGaSP, PCA-PPGaSP, kPCA-PPGaSP (R/RobustGaSP)
   • AE-PPGaSP, VAE-PPGaSP (PyTorch + R/RobustGaSP)
   • BiGP, PCA-BiGP, MTGP (Python/GPyTorch)

4. Benchmark Metrics: Compare model performance and save results

Datasets

Name	Source	Problem	Description
synthetic_100d_function	Generated by 100D function	High-dim input	100D synthetic test function
tsunami_tokushima	Zenodo	High-dim input	Tsunami inundation surrogate model
environment_spill_function	Generated by Environment spill function	High-dim output	Environmental model function
acheron	Figshare + GitHub	High-dim output	Acheron rock avalanche
synthetic_landslide	Figshare + GitHub	High-dim output	Synthetic landslide simulation

Prerequisites

1. Conda Environment Manager
   • conda or
   • micromamba
2. NextFlow
   • Follow the installation instructions

Quick Start

1. Clone repository:

git clone --recurse-submodules https://github.com/gary8564/gpe_bench_nxf.git
git submodule update --init --recursive

2. Run:

nextflow run workflows/main.nf -params-file params.yaml -profile local

Folder Structure

.
├── nextflow.config          # Main NextFlow configuration
├── environment.yml          # Conda environment specification
├── requirements.txt         # Python dependencies
├── README.md
├── main.nf              # Main workflow orchestration
├── img/
│   └── workflow.png         # Pipeline workflow diagram
├── conf/                    # User-defined configuration files
├── envs                     # conda environment yaml files for each process unit
├── modules/                 # Modular NextFlow processes
│   ├── fetch_from_zenodo.nf # Data fetching from Zenodo
│   ├── data_setup_synthetic.nf # Synthetic data generation
│   ├── data_setup_tsunami.nf   # Tsunami data processing
│   ├── preprocessing.nf     # Data standardization and splitting
│   ├── evaluate_exactgp.nf  # Exact GP model training and evaluation
│   ├── evaluate_dkl.nf      # DKL model training and evaluation
│   ├── evaluate_rgasp.nf    # RGaSP model training and evaluation
|   ├── evaluate_pca_rgasp.nf  # PCA-RGaSP model training and evaluation
│   └── benchmark_metrics.nf # Performance comparison and reporting
├── scripts/                 # Implementation scripts corresponding to each module
│   ├── data_setup_synthetic.py  # Synthetic data generation
│   ├── data_setup_tsunami.py    # Tsunami data processing
│   ├── preprocessing.py         # Data preprocessing utilities
│   ├── evaluate_exactgp.py      # Exact GP implementation (Python/GPyTorch)
│   ├── evaluate_dkl.py          # DKL implementation (Python/GPyTorch)
│   ├── evaluate_rgasp.R         # RGaSP implementation (R/RobustGaSP)
|   ├── evaluate_pca_rgasp.nf    # PCA-RGaSP implementation (Python/PCA-RobustGaSP)
│   └── benchmark_metrics.py     # Metrics calculation and comparison

Advanced Usage

Parameter yaml file

Update the params.yaml:

caseStudy: xxxxx                     # name of the case study, should exist in datasets.config
problem_type: high_dim_input         # high_dim_input | high_dim_output
outDir: results                      # name of the output directory
useGPU: false                        # true/false
condaEngine: conda                   # conda/mamba/micromamba
# The following params are used by high_dim_output workflow
# threshold: 5e-06                     # zero-truncation threshold for outputs
# n_components: 10                     # PCA/kPCA components
# latent_dim: 10                       # AE/VAE latent dimension
# Optional quantity of interest for data_setup_landslide: hmax or vmax
# qoi: hmax

If customed datasets are used, make sure that dataset configuration is defined in conf/datasets.config:

params {
  datasets = [
    // ... existing datasets ...

    my_new_study: [
      source: "zenodo",  
      description: "Description of your dataset",
      doi: "10.5281/zenodo.XXXXXXX",  // for Zenodo datasets
      base_url: "https://zenodo.org/records/XXXXXXX",  // for Zenodo datasets
      files: [  // for Zenodo datasets
        "data_file1.csv",
        "data_file2.zip"
      ],
    ]
  ]
}

SLURM:

nextflow run workflows/main.nf \
  -params-file params.yaml \
  -profile slurm

Implementation Details

Benchmark all GP-variants listed below. See more details here.

Model	Language	Library	Kernel	Features
ExactGP	Python	GPyTorch	Matérn 5/2	Exact inference, GPU support
DKL	Python	GPyTorch	Matérn 5/2	Deep kernel learning, GPU support
RGaSP	R	RobustGaSP	Matérn 5/2	Gaussian processes with robust initialization and outlier handling
PCA-RGaSP	R	RobustGaSP	Matérn 5/2	Combine RobustGaSP with PCA for scalability

Advantage of using Nextflow:

1. Language-Agnostic Design

This workflow demonstrates programming language agnosticism in scientific computing pipelines by using HDF5 as cross-language scientific data format so that the data can flow through different stages which might use different OS platform / programming languages / container images.

2. CI-managed Reproducible Environments

• Per-rule/process environment isolation: Each workflow step defines its own Conda environment in envs/, allowing Python/GPyTorch, R/RobustGaSP, and other model-specific dependencies to remain isolated.

• CI-generated lock/pin files: The repository can use CI to regenerate platform-specific explicit Conda specifications whenever envs/*.yml changes. In Nextflow, the explicit .txt lock files can be referenced directly by the conda directive. This makes execution more reproducible.

Comparison between Nextflow and Snakemake

The same benchmark example is used in gpe_bench_smk repo to demonstrate and compare how snakemake and nextflow work.

Aspect	Nextflow	Snakemake
Workflow language	Groovy DSL	Python-based
Paradigm	Channel-based (push; top-down)	File-based DAG (pull; bottom-up)
Config	.config	.yaml
Modules	modules/*.nf (one process per file)	rules/high_dim_input/*.smk, rules/high_dim_output/*.smk (one rule per file)
Intermediate files	Hidden in work/ directory	Visible in output directory

Reference

1. Mölder F, Jablonski KP, Letcher B et al. Sustainable data analysis with Snakemake [version 1; peer review: 1 approved, 1 approved with reservations]. F1000Research 2021, 10:33 (https://doi.org/10.12688/f1000research.29032.1)
2. Henri E. Bal, Jennifer G. Steiner, and Andrew S. Tanenbaum. 1989. Programming languages for distributed computing systems. ACM Comput. Surv. 21, 3 (Sep. 1989), 261–322. https://doi.org/10.1145/72551.72552