ICRL Safety

Constrained fine-tuning of LLM orchestrators using Inverse Constraint Reinforcement Learning on ST-WebAgentBench.

This guide covers everything needed to run the full pipeline on an Alliance / Compute Canada cluster (tested on Fir with SLURM, Apptainer, and H100 GPUs).

The experiment, end to end

One driver runs every stage:

python scripts/run_experiment.py --profile smoke     # minutes, laptop, no GPU
python scripts/run_experiment.py --profile local     # Qwen-0.5B + mock env
python scripts/run_experiment.py --profile cluster   # the real thing

#	Stage	Script	What it produces
0	`preflight`	`run_preflight_checks.py`	fails in seconds on a broken environment
1	`splits`	`make_demo_splits.py`	`<data>/train/.jsonl`, `<data>/eval/_held_out.jsonl`
2	`encode`	`embed_trajectories.py`	`<data>/embeddings/<run>/{safe,unsafe}.pt`
3	`constraint`	`train_constraint.py`	`<ckpt>/<run>/constraint_head.pt`
4	`gate`	`evaluate_constraint.py`	`<ckpt>/<run>/held_out_metrics.json` (AUROC gate)
5	`eval_base`	`evaluate_policy.py`	CuP of the untuned policy
6	`finetune`	`finetune_policy.py`	`<ckpt>/<run>/final` (LoRA adapter)
7	`eval_tuned`	`evaluate_policy.py`	CuP of the tuned policy
8	`plots`	`make_experiment_plots.py`	figures + `report.html`

Paths come from the compute group (local → repo-relative; carleton → $SCRATCH/icrl), overridable with --data-root / --checkpoint-dir / --log-dir.

The summary at the end reports baseline CuP → tuned CuP. A per-run JSON report lands in <logs>/<run_name>_experiment.json.

Useful flags: --stages constraint,gate (subset), --dry-run (print commands), --strict-gate (stop when held-out AUROC misses the threshold — off by default, so the pipeline stays verifiable while the safe demos are still weak), --override key=value (extra Hydra override, repeatable), --pdf (vector figures for LaTeX), --plot-theme dark.

Figures

The plots stage runs last and never fails the job, so a run that died during fine-tuning still gets figures for the stages that finished. Everything lands in <logs>/<run_name>/plots/:

Figure	Shows
`01_constraint_training`	expert vs policy C_θ over ICRL iterations against β, and held-out AUROC climbing toward the gate
`02_gate_heldout`	held-out score distributions and the ROC behind the gate verdict
`03_finetune_dynamics`	task reward, constraint cost vs the ε budget, λ, and CuP / completion / violation rates
`04_cup_comparison`	baseline vs tuned CuP, completion and violation rate — the headline result
`05_violations_by_category`	which ST-WebAgentBench safety dimensions actually fire
`06_stage_timings`	where the wall-clock went

report.html collects all of it — headline tiles, every figure inlined as base64, and the underlying tables — into one self-contained file:

scp cluster:~/icrl/logs/icrl_cluster/plots/report.html .   # then just open it

Regenerate figures at any time without re-running the experiment:

python scripts/make_experiment_plots.py --run-name icrl_cluster --pdf

On the cluster: sbatch scripts/slurm/run_experiment_job.sh (env vars PROFILE, RUN_NAME, STAGES, STRICT_GATE, EXTRA).

Profiles

Profile	Encoder	Policy	Env	Train / held-out tasks	Purpose
`smoke`	`tiny-random-gpt2`	`tiny-random-gpt2`	mock	4 / 2	plumbing only — proves the wiring, says nothing about results
`local`	`Qwen2.5-0.5B`	`Qwen2.5-0.5B-Instruct`	mock	4 / 2	real weights at laptop scale
`cluster`	`Qwen2.5-1.5B`	`Qwen2.5-7B-Instruct`	ST-WebAgentBench	235–249 / 250–254	the experiment

CuP is always measured on tasks the policy did not train on. The policy is 7B rather than the 72B used for demo collection because it has to fit a LoRA fine-tune plus its rollouts on the job's GPUs.

mock is src/environments/mock_environment.py: a deterministic text CRM that emits the same info["safety_report"] contract as the benchmark, so fine-tuning and CuP evaluation run without SuiteCRM, Playwright or a GPU. It is a test fixture, not a benchmark — never report numbers from it.

Choosing the expert demos

--safe-demos / --unsafe-demos accept either a .jsonl of trajectories or a directory of task_*_trace_*.json — the format the SLURM collection job writes to $SCRATCH/trajectories/safe. With neither flag the driver takes data/demos/*.jsonl, falling back to $SCRATCH/trajectories/{safe,unsafe} and printing which it chose.

What ICRL treats as near-optimal expert behaviour matters more than any hyperparameter here:

Source	n	Terminate cleanly	Notes
`data/demos/safe.jsonl`	81	0	Qwen-72B rollouts; 79/81 have `reward=0.0`. Safe but not near-optimal — half of what ICRL assumes.
`$SCRATCH/trajectories/safe`	1	1	The CuP=1 trace from job 45204272. Correct shape (confirm → delete, 6 steps), but one task is not a split.
`data/demos/webarena_raw.jsonl`	177	177	WebArena human traces. Humans finish and stop.

python scripts/run_experiment.py --profile cluster \
    --safe-demos data/demos/webarena_raw.jsonl

preflight reports trajectory count, distinct tasks and clean-termination count per source, so this shows up before training rather than after.

Running without the browser

pytest tests/test_pipeline_end_to_end.py -q

Pins the CuP measurement to scripted policies: confirm-then-delete scores CuP=1, delete-immediately scores CuP=0, do-nothing scores CuP=0. Without these, a CuP of 0.000 from a weak model is indistinguishable from a broken metric.

The same file also guards the contract against the real benchmark (skipped when ST-WebAgentBench is not importable): that answer() is emitted at module level, that the env is built with an action_mapping, that safety_report parses in both its nested and flattened shapes, and that the collection job's trace format loads.

Cluster runbook

Nothing about the cluster is hardcoded any more — allocation, GPU type and partition are discovered, then passed on the sbatch command line (#SBATCH directives are literal text and cannot read environment variables).

# 0. Discover what THIS cluster offers — accounts, GPU types, modules, quota
bash scripts/discover_cluster_resources.sh
export ICRL_ACCOUNT=aip-...        # from the output
export ICRL_GPU=l40s:1             # or h100:1 — from the output
# If your allocation has no usable /scratch:
#   export SCRATCH=/project/<alloc>/$USER

# 1. One-time setup
export GITHUB_USER=<you> REPOS_ROOT=$SCRATCH
bash scripts/setup_cluster.sh
cp .env.example .env && $EDITOR .env

# 2. Prefetch models — LOGIN NODE ONLY (compute nodes have no internet)
source scripts/start_session.sh
python scripts/prefetch_models.py --profile cluster

# 3. Front half: no browser, no CRM, no SuiteCRM needed
bash scripts/submit_experiment.sh \
    --stages preflight,splits,encode,constraint,gate,plots

# 4. Full run — needs SuiteCRM up on the login node first
bash scripts/start_suitecrm_apptainer.sh
echo "WA_SUITECRM=http://$(hostname):8080/public" >> .env
bash scripts/submit_experiment.sh

Variable	Meaning
`ICRL_ACCOUNT`	allocation to charge (required)
`ICRL_GPU`	`l40s:1`, `h100:2`, a bare count, or `0` for CPU-only
`ICRL_PARTITION`	only if the cluster needs an explicit one
`ICRL_TIME` / `ICRL_MEM` / `ICRL_CPUS`	defaults `12:00:00` / `64G` / `8`
`PROFILE` / `RUN_NAME`	default `cluster` / `icrl_cluster`

DRY_RUN=1 bash scripts/submit_experiment.sh prints the sbatch line without submitting.

Offline compute nodes. Alliance compute nodes cannot reach the internet, so submit_experiment.sh exports HF_HUB_OFFLINE=1 and every model must already sit in $HF_HOME ($SCRATCH/hf_cache). prefetch_models.py --check verifies this, and preflight fails with the exact prefetch command if a model is missing.

.env must set WA_SUITECRM, and also GITLAB and SHOPPING_ADMIN — ST-WebAgentBench validates every site URL when it loads and refuses to start if any is missing, even for SuiteCRM-only tasks. Point them at SuiteCRM if you have no other instances; nothing in the easy tier dereferences them. scripts/slurm/job_environment.sh exports .env to every stage.

What runs where

ST-WebAgentBench tasks operate a real CRM web app (SuiteCRM) through Playwright inside SLURM jobs. The LLM actor runs via vLLM on the job's GPUs.

flowchart LR
  subgraph login["Login node (persistent)"]
    CRM["SuiteCRM + MariaDB\n(Apptainer instances)"]
  end
  subgraph compute["Compute node (per job)"]
    vLLM["vLLM\nQwen-72B"]
    PW["Playwright\n(headless browser)"]
  end
  CRM -->|"WA_SUITECRM\nhttp://loginN:8080/public"| PW
  vLLM --> PW
  PW --> OUT["Trajectory JSON\n/scratch/$USER/trajectories/"]

Component	Where	Why
Python venv + code	`/scratch/$USER/venvs/icrl_v4`	Fast I/O, shared across jobs
SuiteCRM + MariaDB	Login node (Apptainer)	Stays up across jobs; compute nodes reach it over the cluster network
vLLM + Playwright	Compute node (inside SLURM job)	Needs GPUs; browser must co-locate with the model
Trajectories	`/scratch/$USER/trajectories/`	Persistent output

Dependencies (third-party repos)

Repo	Upstream	Our use
BrowserGym	ServiceNow	Browser env core
ST-WebAgentBench	segev-shlomov	Benchmark tasks + evaluators

We maintain forks with small compatibility patches. Never push directly to upstream.

A note on /scratch vs /project: every path below assumes /scratch/$USER. On some Alliance accounts (e.g. aip-s2ganapa) there's no usable /scratch quota and everything instead lives under /project/aip-s2ganapa/$USER. All scripts read the SCRATCH env var (defaulting to /scratch/$USER), so on those accounts export it before sourcing anything:
export SCRATCH=/project/aip-s2ganapa/$USER
export ICRL_ROOT=/project/aip-s2ganapa/$USER/icrl
Do this once per session (or add to ~/.bashrc) before source scripts/start_session.sh — otherwise it looks for the repo/venv at ~/icrl and /scratch/$USER/... and fails with No such file or directory.

Prerequisites

Alliance cluster account with GPU allocation (e.g. def-s2ganapa)
SSH access to login node
GitHub account with forks of BrowserGym and ST-WebAgentBench
API keys: OPENROUTER_API_KEY (required for collection), HUGGINGFACE_TOKEN (for model download / fine-tuning)

Modules used

module load gcc python/3.12 arrow/23.0.1 cuda/12.1 cudnn/8.9   # Python jobs
module load apptainer/1.4.5                                     # SuiteCRM

Step 1 — Clone and one-time setup

1a. Fork upstream repos (browser, once)

Fork BrowserGym
Fork ST-WebAgentBench

1b. Clone icrl and run setup

git clone git@github.com:YOUR_USER/icrl.git ~/icrl
cd ~/icrl

export GITHUB_USER=YOUR_USER
export REPOS_ROOT=$HOME                    # clones BrowserGym + ST-WebAgentBench here
bash scripts/setup_cluster.sh

setup_cluster.sh will:

Clone your forks of BrowserGym and ST-WebAgentBench
Create venv at /scratch/$USER/venvs/icrl_v4
Install icrl, BrowserGym, ST-WebAgentBench, vLLM, Playwright deps
Download NLTK data
Write activate_icrl.sh with PYTHONPATH exports
Verify 375 ST-WebAgentBench tasks register

Login-node tip: Playwright browser download can be slow on login nodes. To skip and install later on a compute node:

SKIP_PLAYWRIGHT=1 bash scripts/setup_cluster.sh
# then on a GPU node or interactive session:
source /scratch/$USER/venvs/icrl_v4/bin/activate
playwright install chromium

1c. Activate environment (every session)

# If your account uses /project instead of /scratch, export SCRATCH/ICRL_ROOT first — see note above.
source /scratch/$USER/venvs/icrl_v4/bin/activate
source /scratch/$USER/venvs/icrl_v4/bin/activate_icrl.sh
cd ~/icrl

Or, simpler, from the repo root: source scripts/start_session.sh — it does the module load, venv activate, PYTHONPATH export, and a SuiteCRM reachability check in one step (respects SCRATCH/ICRL_ROOT overrides from the note above).

activate_icrl.sh sets:

Variable	Default
`ICRL_ROOT`	`~/icrl`
`STWEBAGENT_ROOT`	`$REPOS_ROOT/ST-WebAgentBench`
`BROWSERGYM_ROOT`	`$REPOS_ROOT/BrowserGym`
`PYTHONPATH`	`icrl` + `icrl/src/gridworld`

Step 2 — Configure environment variables

cp .env.example .env
cp $STWEBAGENT_ROOT/.env.example $STWEBAGENT_ROOT/.env

Edit ~/icrl/.env:

OPENROUTER_API_KEY=sk-or-...          # required for trajectory collection
OPENAI_API_KEY=sk-...                 # optional
HUGGINGFACE_TOKEN=hf_...              # required for HF model weights
WA_SUITECRM=http://login3:8080/public # set after Step 3 (use your login node hostname)

Edit $STWEBAGENT_ROOT/.env (benchmark reads web-app URLs from here too):

WA_SUITECRM=http://login3:8080/public

Replace login3 with the hostname of the login node where SuiteCRM runs (hostname on that node).

SLURM jobs load ~/icrl/.env automatically (scripts/slurm/job_environment.sh exports it to every stage). With WA_SUITECRM set, jobs connect directly to your login-node SuiteCRM instance.

Step 3 — SuiteCRM on the login node (Apptainer)

SuiteCRM is the CRM web app the benchmark clicks through. Easy-tier tasks use task IDs 235–254 (SuiteCRM only).

Why Apptainer?

No subuid / rootless Podman headaches on Alliance login nodes
SIF images persist on /scratch
One persistent CRM instance shared by all SLURM jobs

Image note (June 2026)

Bitnami removed public.ecr.aws/bitnami/* on 2026-06-10. Use frozen legacy images:

Service	Docker image	SIF path
MariaDB	`bitnamilegacy/mariadb:11.4`	`/scratch/$USER/apptainer/mariadb.sif`
SuiteCRM	`bitnamilegacy/suitecrm:8`	`/scratch/$USER/apptainer/suitecrm.sif`

3a. Pull SIF images (one-time, ~30 min)

module load apptainer/1.4.5
mkdir -p /scratch/$USER/apptainer/tmp

export APPTAINER_TMPDIR=/scratch/$USER/apptainer/tmp

apptainer pull /scratch/$USER/apptainer/mariadb.sif \
  docker://bitnamilegacy/mariadb:11.4

apptainer pull /scratch/$USER/apptainer/suitecrm.sif \
  docker://bitnamilegacy/suitecrm:8

3b. Start SuiteCRM (login node)

Important flags:

Use apptainer instance **run** — not instance start (start only runs appinit, not MariaDB/Apache)
Add --writable-tmpfs — SIF images are read-only; without this MariaDB fails with Read-only file system

Helper script (recommended):

module load apptainer/1.4.5
bash scripts/start_suitecrm_apptainer.sh

The script waits for SuiteCRM to respond (up to 10 min) before returning. First boot initialises the database and takes ~10 min; subsequent starts take ~1 min.

Manual commands:

module load apptainer/1.4.5
mkdir -p /scratch/$USER/suitecrm/{mariadb,app}

apptainer instance run \
  --writable-tmpfs \
  --bind /scratch/$USER/suitecrm/mariadb:/bitnami/mariadb \
  --env ALLOW_EMPTY_PASSWORD=yes \
  --env MARIADB_USER=bn_suitecrm \
  --env MARIADB_DATABASE=bitnami_suitecrm \
  --env MARIADB_PASSWORD=bitnami123 \
  /scratch/$USER/apptainer/mariadb.sif mariadb

sleep 30

apptainer instance run \
  --writable-tmpfs \
  --bind /scratch/$USER/suitecrm/app:/bitnami/suitecrm \
  --env SUITECRM_DATABASE_HOST=127.0.0.1 \
  --env SUITECRM_DATABASE_PORT_NUMBER=3306 \
  --env SUITECRM_DATABASE_USER=bn_suitecrm \
  --env SUITECRM_DATABASE_NAME=bitnami_suitecrm \
  --env SUITECRM_DATABASE_PASSWORD=bitnami123 \
  --env ALLOW_EMPTY_PASSWORD=yes \
  /scratch/$USER/apptainer/suitecrm.sif suitecrm

until curl -sf http://localhost:8080 > /dev/null 2>&1; do
  sleep 15; echo "$(date +%H:%M:%S) waiting..."
done
echo "SuiteCRM up at http://$(hostname):8080"

First boot takes ~10 minutes (database initialisation + SuiteCRM install wizard). Subsequent starts take ~30 seconds.

3c. Set WA_SUITECRM

echo "WA_SUITECRM=http://$(hostname):8080/public" >> ~/icrl/.env
# also update $STWEBAGENT_ROOT/.env with the same URL

3d. Verify SuiteCRM

curl -sf http://localhost:8080/public -o /dev/null -w 'HTTP %{http_code}\n'
apptainer instance list          # should show mariadb + suitecrm
ss -tlnp | grep -E '3306|8080'  # MariaDB and Apache listening

Default SuiteCRM credentials (Bitnami image): user / bitnami

3e. Manage instances

bash scripts/start_suitecrm_apptainer.sh            # start + wait for HTTP (default)
bash scripts/start_suitecrm_apptainer.sh --status   # list running instances
bash scripts/start_suitecrm_apptainer.sh --stop     # stop both

After a login-node reboot

module load apptainer/1.4.5
bash scripts/start_suitecrm_apptainer.sh

Data persists in /scratch/$USER/suitecrm/{mariadb,app} — you do not need to re-pull SIF images or re-initialise the DB.

Optional: load demo data

After first boot, you can seed the CRM with benchmark demo data:

# if using Docker locally; for Apptainer, exec into the mariadb instance:
apptainer exec instance://mariadb \
  mysql -u bn_suitecrm -pbitnami123 bitnami_suitecrm \
  < $STWEBAGENT_ROOT/suitecrm_setup/init-db/demo_data.sql

Step 4 — Verify the full stack

Run on the login node (with SuiteCRM up):

source /scratch/$USER/venvs/icrl_v4/bin/activate
source /scratch/$USER/venvs/icrl_v4/bin/activate_icrl.sh
cd ~/icrl

# 375 tasks registered
python -c "import browsergym.stwebagentbench, gymnasium as gym; \
  print(len([e for e in gym.envs.registry if 'STWebAgent' in e]), 'tasks')"

# icrl env wrapper
python -c "from icrl.envs.stwebagent import STWebAgentEnv; print('OK')"

# No browser or GPU needed — resolved config only
python scripts/collect_trajectories.py \
  --config configs/trajectory_collection/stwebagentbench_expert.yaml --dry-run

# Live smoke collection (SuiteCRM must be reachable; OpenRouter backend)
python scripts/collect_trajectories.py \
  --config configs/trajectory_collection/stwebagentbench_expert.yaml --smoke

Step 5 — Submit SLURM jobs

All scripts/slurm/*.sh scripts use account def-s2ganapa and write logs to logs/slurm/. Edit #SBATCH --account= in each script if your allocation differs.

cd ~/icrl
mkdir -p logs/slurm

Session checklist

Every time before submitting:

source /scratch/$USER/venvs/icrl_v4/bin/activate
source /scratch/$USER/venvs/icrl_v4/bin/activate_icrl.sh
cd ~/icrl

# Confirm SuiteCRM is up (on login node)
curl -sf http://login3:8080/public -o /dev/null && echo "CRM OK" || echo "CRM DOWN — run start_suitecrm_apptainer.sh"

5a. Dry run (no GPU, no browser)

Validates imports, .env, and the resolved config without starting vLLM:

DRY_RUN=1 CONFIG=configs/trajectory_collection/stwebagentbench_expert.yaml \
  sbatch --gres= --mem=4G --time=00:10:00 scripts/slurm/collect_trajectories_job.sh
tail -f logs/slurm/icrl-collect-trajectories_*.out

5b. Trajectory collection

One wrapper for every collection run — the config defines benchmark, tasks, model, prompt, and keep rule (see docs/trajectory-collection.md). GPU count on the sbatch line must match the config's model.tensor_parallel.

# Expert: Qwen-72B + policy prompt, keep only CuP=1 (4× GPU, tensor-parallel=4)
CONFIG=configs/trajectory_collection/stwebagentbench_expert.yaml \
  sbatch --account=$ICRL_ACCOUNT --gres=gpu:h100:4 scripts/slurm/collect_trajectories_job.sh

# Unsafe: Qwen-7B, no safety prompt, keep any violating episode (1× GPU)
CONFIG=configs/trajectory_collection/stwebagentbench_unsafe.yaml \
  sbatch --account=$ICRL_ACCOUNT --gres=gpu:h100:1 scripts/slurm/collect_trajectories_job.sh

# Subset without editing the config:
OVERRIDES="benchmark.task_ids=[235,236,237]" CONFIG=... sbatch ...

Output: $SCRATCH/trajectories/stwebagentbench/{expert,unsafe}/task_*_trace_*.json plus manifest.json (resolved config) and summary.csv per directory.

5d. Other pipeline jobs

Script	GPUs	Purpose
`scripts/slurm/embed_trajectories_job.sh`	1	Embed collected trajectories
`scripts/slurm/train_constraint_job.sh`	1	Train constraint encoder
`scripts/slurm/finetune_policy_job.sh`	2	Fine-tune orchestrator
`scripts/slurm/build_chain_of_thought_dataset_job.sh`	1	Build CoT dataset
`scripts/slurm/finetune_chain_of_thought_policy_job.sh`	2	CoT fine-tuning
`scripts/slurm/finetune_policy_array_job.sh`	2 × 9	Hyperparameter sweep

Monitor jobs

squeue -u $USER
tail -f logs/slurm/icrl-gen_*.out
tail -f logs/slurm/icrl-gen_*.err
scancel JOBID

Directory layout on `/scratch`

/scratch/$USER/
├── venvs/icrl_v4/              Python venv
├── apptainer/
│   ├── mariadb.sif             MariaDB image (~123 MB)
│   ├── suitecrm.sif            SuiteCRM image (~306 MB)
│   ├── suitecrm_overlay.img    Persistent ext3 overlay for SuiteCRM (2 GB, auto-created)
│   └── tmp/                    Apptainer build temp
├── suitecrm/
│   ├── mariadb/                MariaDB data (persistent)
│   └── app/                    SuiteCRM data (persistent)
├── trajectories/
│   └── safe/                   Collected trajectory JSON
└── hf_cache/                   HuggingFace model cache (vLLM)

Troubleshooting

SuiteCRM

Symptom	Fix
`instance mariadb already exists`	`apptainer instance stop mariadb; apptainer instance stop suitecrm`
MariaDB `Read-only file system`	Add `--writable-tmpfs` to `instance run`
Only `appinit` running, no `mariadbd`	Use `instance run`, not `instance start`
`curl localhost:8080` fails right after start	SuiteCRM is still booting — the script now waits automatically; check `~/.apptainer/instances/logs/$HOSTNAME/$USER/suitecrm.out` if it times out
`No space left on device` in suitecrm.err	The bitnami SIF has a large Angular cache baked in; the tiny tmpfs overlay fills up when the entrypoint tries to delete it. Fixed: `start_suitecrm_apptainer.sh` now uses a 2 GB persistent ext3 overlay (`/scratch/$USER/apptainer/suitecrm_overlay.img`) instead of `--writable-tmpfs`. If corrupted, recreate with `bash scripts/start_suitecrm_apptainer.sh --reset-overlay`.
`Device or resource busy` on `.angular`	Caused by an old version of the script that bind-mounted `.angular` to scratch — the mountpoint couldn't be rmdir'd. Update to the current script which uses `--overlay`.
Compute node can't reach CRM	Use login-node hostname in `WA_SUITECRM`, not `localhost`; confirm port 8080 reachable from compute nodes
`apptainer pull` slow / fails on `/scratch`	Set `APPTAINER_TMPDIR=/scratch/$USER/apptainer/tmp`

SLURM

Symptom	Fix
`Batch job submission failed: account`	Edit `#SBATCH --account=` in the script to your allocation
Job exits immediately at CRM startup	Set `WA_SUITECRM` in `.env` so job skips inline CRM boot
vLLM OOM	Reduce model size or increase `--gres=gpu:h100:N` and `TP_SIZE`
Playwright browser not found	Run `playwright install chromium` inside venv (on a node with network)

Python

Symptom	Fix
`typing_extensions` import errors	`pip install "typing_extensions>=4.13.0" --force-reinstall --no-deps`
0 ST-WebAgent tasks registered	Re-run `setup_cluster.sh`; check `stwebagentbench.pth` in site-packages
`ModuleNotFoundError: icrl`	`source activate_icrl.sh` to set `PYTHONPATH`
`~/icrl/scripts/start_session.sh: No such file or directory`	Repo/venv live under `/project/...` on this account, not `~/icrl` and `/scratch/$USER`. `cd` into the actual repo dir and export `SCRATCH`/`ICRL_ROOT` first — see the `/scratch` vs `/project` note near the top.

Logs

# Apptainer instance logs
ls ~/.apptainer/instances/logs/$(hostname)/$USER/
tail -f ~/.apptainer/instances/logs/$(hostname)/$USER/suitecrm.out

# SLURM job logs
tail -f logs/slurm/icrl-gen_*.out
tail -f logs/slurm/vllm_*.log

Local development (laptop / Docker)

For development off-cluster, use Docker Compose:

bash scripts/start_suitecrm_docker.sh
# first boot — load demo data:
docker exec -i suitecrm_setup-mariadb-1 \
  mysql -u bn_suitecrm -pbitnami123 bitnami_suitecrm \
  < $STWEBAGENT_ROOT/suitecrm_setup/init-db/demo_data.sql

Set WA_SUITECRM=http://localhost:8080/public in .env.

Fork workflow

Remotes

Remote	Points to
`origin` / `fork`	your GitHub fork
`upstream`	original repo

export GITHUB_USER=YOUR_USER
export REPOS_ROOT=$HOME
bash scripts/setup_fork_remotes.sh

icrl-specific patches (ST-WebAgentBench fork)

stwebagentbench/browser_env/custom_env.py — TEXT_MAX_LENGTH import fix (browsergym 0.14+)
Explicit pydantic / typing_extensions deps for Compute Canada venvs

Sync with upstream

cd $STWEBAGENT_ROOT
git fetch upstream && git merge upstream/main && git push fork main

Repo layout

~/icrl/                         this repo
~/BrowserGym/                   fork of ServiceNow/BrowserGym
~/ST-WebAgentBench/             fork of segev-shlomov/ST-WebAgentBench

icrl/
  src/                          pipeline-stage packages (see CLAUDE.md)
  src/gridworld/                gridworld ICRL reference implementation
  configs/                      Hydra configs, one directory per pipeline stage
  scripts/                      entry points, setup_cluster.sh, start_suitecrm_apptainer.sh
  scripts/slurm/                SLURM job templates + job_environment.sh
  scratch/                      one-off / throwaway scripts, not part of the pipeline

Requirements files

File	Use
`requirements.txt`	Full install (ML + browser + vLLM)
`requirements-no-agentlab.txt`	Lighter install without agentlab/gradio/ray
`requirements-browser.txt`	BrowserGym + ST-WebAgentBench deps only

Tests

export PYTHONPATH=.:src/gridworld
pytest src/gridworld/tests/unit/ -q
pytest tests/ --ignore=tests/test_reasoning_trace.py -q

Quick reference — full cluster bootstrap

# If your account uses /project instead of /scratch (e.g. aip-s2ganapa), do this first:
# export SCRATCH=/project/aip-s2ganapa/$USER
# export ICRL_ROOT=/project/aip-s2ganapa/$USER/icrl

# === ONE TIME ===
git clone git@github.com:YOUR_USER/icrl.git ~/icrl && cd ~/icrl
export GITHUB_USER=YOUR_USER REPOS_ROOT=$HOME
bash scripts/setup_cluster.sh
cp .env.example .env && $EDITOR .env

module load apptainer/1.4.5
mkdir -p /scratch/$USER/apptainer/tmp
export APPTAINER_TMPDIR=/scratch/$USER/apptainer/tmp
apptainer pull /scratch/$USER/apptainer/mariadb.sif docker://bitnamilegacy/mariadb:11.4
apptainer pull /scratch/$USER/apptainer/suitecrm.sif docker://bitnamilegacy/suitecrm:8
bash scripts/start_suitecrm_apptainer.sh   # starts + waits until HTTP 200
echo "WA_SUITECRM=http://$(hostname):8080/public" >> ~/icrl/.env

# === EVERY SESSION ===
source scripts/start_session.sh   # or manually:
source /scratch/$USER/venvs/icrl_v4/bin/activate
source /scratch/$USER/venvs/icrl_v4/bin/activate_icrl.sh
cd ~/icrl

# === RUN COLLECTION ===
mkdir -p logs/slurm
OVERRIDES="benchmark.task_ids=[235,236]" CONFIG=configs/trajectory_collection/stwebagentbench_expert.yaml \
  sbatch --account=$ICRL_ACCOUNT --gres=gpu:h100:4 scripts/slurm/collect_trajectories_job.sh
tail -f logs/slurm/icrl-gen_*.out

Name		Name	Last commit message	Last commit date
Latest commit History 46 Commits
configs		configs
docs		docs
logs/slurm		logs/slurm
scratch		scratch
scripts		scripts
src		src
tests		tests
.env.example		.env.example
.gitignore		.gitignore
CLAUDE.md		CLAUDE.md
README.md		README.md
RESEARCH_LOG.md		RESEARCH_LOG.md
requirements-browser.txt		requirements-browser.txt
requirements-no-agentlab.txt		requirements-no-agentlab.txt
requirements.txt		requirements.txt
setup.py		setup.py
suitecrm-compose.yaml		suitecrm-compose.yaml

Folders and files

Latest commit

History

Repository files navigation