Complete Guide: From Demo Collection to CLIP Training

Overview

This guide covers everything from collecting multi-mission demos to training CLIP on the expanded dataset.

Part : Full Collection (Some Days + Hours xD)

Recommended Order (easiest → hardest):

1. BabyAI-GoToObj-v0 Start here

   • ~12 variants
   • Minimal rejections
   • Fast episodes

2. BabyAI-Open-v0

   • ~8 door variants
   • Some rejections

3. BabyAI-PickupLoc-v0

   • ~10-15 pickup variants
   • More rejections

4. BabyAI-PutNextLocal-v0

   • ~15-20 placement variants
   • Many rejections

5. BabyAI-GoToSeqS5R2-v0

   • ~20+ sequential variants
   • Many rejections

6. BabyAI-SynthS5R2-v0

   • ~20+ composition variants
   • Many rejections

7. BabyAI-BossLevel-v0 🔥 Hardest

   • Limit to 30 most common missions
   • Extreme rejections

Collection Commands:

# Collect 1000 demos per variant for each environment
uv run test_simple_env.py BabyAI-GoToObj-v0
uv run test_simple_env.py BabyAI-Open-v0
uv run test_simple_env.py BabyAI-PickupLoc-v0
# etc.

# For BossLevel, use special handling:
# - Collect diverse sample
# - Or limit to top 30 missions

---

Part 4: Update Curriculum

Generate Curriculum Entries:

# After collecting demos, generate curriculum
python generate_curriculum_entries.py > new_curriculum.txt

---

Part 5: Train IL on Expanded Curriculum

# Train IL policies on ALL expanded tasks
python main_il.py --phase curriculum --device cuda

# This will take MUCH longer than before (100+ tasks vs 17)
# But you'll have way more base tasks for CLIP!

---

Part 6: Collect Trajectories

# After IL training completes, collect trajectories
python main_il.py --phase il_trajectories --device cuda

# Collects trajectories from ALL 100+ trained policies
# Each policy contributes to CLIP training

---

Part 7: Train CLIP on Expanded Dataset

# CLIP now trains on 100+ (instruction, policy) pairs!
python main_il.py --phase clip --epochs 100 --device cuda

# CLIP learns:
# - "go to red ball" and "go to blue box" have similar policies
# - Task structure matters more than specific objects
# - Color/object are surface variations

---

Part 8: Evaluate Transfer

# Test transfer to new tasks
python main_il.py --phase transfer --seeds 3 --device cuda

---

Expected Results

Before Multi-Mission Expansion:

• Base tasks: 17
• CLIP training pairs: 17
• Transfer: Limited to exact instruction matches

After Multi-Mission Expansion:

• Base tasks: 100-150
• CLIP training pairs: 100-150
• Transfer: Generalizes across color/object variations

Example Transfer Improvement:

Target task: "go to the purple key"

Before:

• No similar base task
• Random initialization
• Transfer success: ~30%

After:

• Has "go to red key", "go to blue key", "go to green key" in base tasks
• CLIP learns color is surface variation
• Transfer success: ~75%