Frontier Learning

This repository includes synthetic regression, MIMIC-IV clinical-note, and VisDA image-transfer experiments.

Data layout

frontier_learning/
├── src/                              # experiment scripts
├── object/                           # VisDA utilities
├── Datasets/
│   ├── sim/
│   ├── mimiciv/3.1/
│   │   ├── hosp/{admissions,patients,diagnoses_icd}.csv.gz
│   │   └── icu/icustays.csv.gz
│   ├── mimic-iv-note/2.2/note/
│   │   ├── discharge.csv.gz
│   │   └── radiology.csv.gz
│   └── mimiciv_icu_domains_with_notes/   # generated
├── VisDA_Dataset/
│   ├── {clipart,infograph,painting,quickdraw,real,sketch}/
│   └── <domain>_{train,test}.txt
└── results_*/                        # outputs and checkpoints

Synthetic regression

# Generate data
python src/data_generation_sim.py --seed 1 --out_dir Datasets/sim/rep_1

# Train and evaluate
python src/analysis_frontier_learning_mlp_sim.py \
  --seed 1 --data_dir Datasets/sim/rep_1 \
  --out_dir results_rep100_sim/rep_1

# Aggregate rep_<seed> directories
python src/summary_sim.py \
  --results_root results_rep100_sim --seeds $(seq 0 99) \
  --out_dir results_rep_summary

Use one data/output directory per seed. Generated data are four .npz files (X/Y_src/tgt) containing train, validation, and test arrays.

MIMIC-IV notes

# Build MICU/CVICU/SICU/CCU/TSICU datasets
python src/data_generation_mimiciv_domians_withNotes.py

# Train required source models and create embedding caches
for DOMAIN in MICU CVICU SICU CCU; do
  python src/analysis_frontier_learning_mimiciv_sources.py \
    --data_root Datasets/mimiciv_icu_domains_with_notes \
    --out_root results_clinicalbert_chunk_source \
    --source_domain "$DOMAIN" --seed 42
done

# Run MICU -> CVICU Frontier Learning
python src/analysis_frontier_learning_mimicNotes_micu_cvicu.py \
  --source_root results_clinicalbert_chunk_source \
  --finetune_root results_clinicalbert_chunk_finetune \
  --direct_root results_clinicalbert_chunk_direct \
  --out_dir results_mimiciv_frontier_learning_cvicU \
  --target_domain CVICU --seed 42

The main output is frontier_learning_results.json. For repeated runs, store it under results_rep_mimicNotes/seed_<seed>/ensemble/, then run python src/summary_mimicNotes.py.

VisDA Painting

Use example_visda.ipynb to download VisDA, then run:

# Train source models
python src/analysis_resnet_visda_source.py \
  --data_root VisDA_Dataset --output_root visda_source_checkpoints \
  --domains clipart real sketch --gpu_id 0

# Run Frontier Learning
python src/analysis_frontier_learning_visda_painting.py \
  --data_root VisDA_Dataset \
  --source_ckpt_root visda_source_checkpoints \
  --out_dir visda_painting_10seed_results/seed_0 \
  --gpu_id 0 --seed 0

# Aggregate seed_<seed> directories
python src/summary_visda.py \
  --result_root visda_painting_10seed_results --plot_scatter_box

Each VisDA seed directory contains summary_results.csv, checkpoints, cached features, and curves. Never share writable output directories across seeds or configurations.