GIFT

Official implementation for Target-Driven Policy Optimization for Sequential Counterfactual Outcome Control.

This repository contains code for learning goal-conditioned intervention policies from observational temporal data. The main method, GIFT, is evaluated on semi-synthetic tumor simulation tasks and MIMIC-III based synthetic treatment-outcome benchmarks, together with several counterfactual sequence modeling and policy optimization baselines.

Overview

GIFT addresses sequential counterfactual outcome control: given a patient's current history and a desired target outcome, the goal is to learn adaptive intervention policies that steer future trajectories toward the target. The codebase includes:

• GIFT training and evaluation code.
• Baselines including VCIP, RMSN, CRN, CT, ACTIN, and SCRL.
• Hydra configuration files for datasets, models, and hyperparameters.
• Experiment runners for main comparisons, ablations, sensitivity studies, and complexity analysis.
• Analysis scripts for generating tables, figures, and case studies.

Repository Structure

configs/                 Hydra configs for datasets, models, and hyperparameters
experiments/             Training, batch experiment, data generation, and analysis scripts
src/gift/                GIFT agents, buffers, model modules, and utilities
src/baselines/           Baseline implementations
src/data/                Dataset loaders and synthetic data generators
analysis_output/         Pre-generated analysis tables and figures
requirements.txt         Python dependencies
ct_requirements.txt      Alternative environment requirements for CT-style baselines
run.sh                   Example batch experiment commands
train.sh                 Example single-model training script

Data

Large data files are not included in this repository. The data/ directory is intentionally ignored by Git.

For MIMIC-III based experiments, the default config expects the processed HDF5 file at:

data/processed/all_hourly_data.h5

Tumor simulation data can be generated on demand by the training pipeline from the corresponding Hydra dataset configuration.

Installation

Create and activate a Python environment, then install dependencies:

pip install -r requirements.txt

Some baselines may require a separate environment. The provided scripts assume two conda environments:

• ct for GIFT, RMSN, CRN, CT, and SCRL-style runs.
• vcip for VCIP and ACTIN runs.

Adjust train.sh or experiments/run_experiments.py if your environment names differ.

Quick Start

Run a single tumor experiment with GIFT:

CUDA_VISIBLE_DEVICES=0 python experiments/train.py \
  +dataset=tumor \
  +model=gift \
  +hparam/gift/tumor=4* \
  exp.seed=10 \
  exp.logging=False \
  exp.max_epochs=15 \
  model.name=gift \
  exp.test=False \
  dataset.num_patients.train=1000 \
  exp.load_data=False \
  exp.load_model=False

Run a single MIMIC-III synthetic experiment:

CUDA_VISIBLE_DEVICES=0 python experiments/train.py \
  +dataset=mimic \
  +model=gift \
  +hparam/gift=mimic \
  exp.seed=10 \
  exp.logging=False \
  exp.max_epochs=30 \
  model.name=gift \
  exp.test=False \
  dataset.max_number=500 \
  exp.load_data=True

Batch Experiments

The batch runner supports multi-GPU scheduling and several experiment groups:

python experiments/run_experiments.py --experiment main_comparison --tasks-per-gpu 5
python experiments/run_experiments.py --experiment goal_threshold --tasks-per-gpu 5
python experiments/run_experiments.py --experiment gift_ablation --tasks-per-gpu 5
python experiments/run_experiments.py --experiment epsilon_study --tasks-per-gpu 5 --allowed-gpus 0

Available experiment names include:

main_comparison
goal_threshold
train_size
baseline_k
gift_ablation
complexity
epsilon_study
epsilon1_study
goal_coupling_study
highdim_comparison
discrete_comparison
unobserved_comparison
all

Outputs

Training and evaluation outputs are written under results/ by default. Depending on the experiment, the pipeline saves:

• Raw CSV metrics.
• Aggregated JSON summaries.
• Checkpoints.
• TensorBoard logs.
• Case-study trajectories and complexity statistics.

The analysis_output/ directory contains generated tables and figures used for reporting results.

Citation

@inproceedings{
anonymous2026targetdriven,
title={Target-Driven Policy Optimization for Sequential Counterfactual Outcome Control},
author={Xin Wang and Xiangyu Zhang and Shengfei Lyu and Huanhuan Chen},
booktitle={Forty-third International Conference on Machine Learning},
year={2026},
url={https://openreview.net/forum?id=2OtY6CfcFs}
}