Lifelong Language-Conditioned Robotic Manipulation Learning

Traditional language-conditioned manipulation agent adaptation to new manipulation skills leads to catastrophic forgetting of old skills, limiting dynamic scene practical deployment. In this paper, we propose SkillsCrafter, a novel robotic manipulation framework designed to continually learn multiple skills while reducing catastrophic forgetting of old skills. Specifically, we propose a Manipulation Skills Adaptation to retain the old skills knowledge while inheriting the shared knowledge between new and old skills to facilitate learning of new skills. Meanwhile, we perform the singular value decomposition on the diverse skill instructions to obtain common skill semantic subspace projection matrices, thereby recording the essential semantic space of skills. To achieve forget-less and generalization manipulation, we propose a Skills Specialization Aggregation to compute inter-skills similarity in skill semantic subspaces, achieving aggregation of the previously learned skill knowledge for any new or unknown skill. Extensive simulator and real-world experiments demonstrate the effectiveness and superiority of our SkillsCrafter.

---

Installation

1. Clone this repository

git clone https://github.com/lifelong-SkillsCrafter/code

2. Simulation Environment Installation

conda create -n crafter python=3.8.1 -y
cd SkillsCrafter/sim
git clone https://github.com/stepjam/PyRep.git
cd PyRep
pip install -r requirements.txt
pip install .

Add the following to your ~/.bashrc file: (NOTE: the 'EDIT ME' in the first line)

export COPPELIASIM_ROOT=<EDIT ME>/PATH/TO/COPPELIASIM/INSTALL/DIR
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$COPPELIASIM_ROOT
export QT_QPA_PLATFORM_PLUGIN_PATH=$COPPELIASIM_ROOT

pip install -r requirements.txt
pip install .

3. Install additional packages for training cases

cd SkillsCrafter
pip install -e ".[train]"
pip install flash-attn --no-build-isolation

---

Prepare Simulation Data

1. Generate RLBench Demos

cd SkillsCrafter/sim/
bash data.sh

2. Adapt the Format

We provide a script to make it easier to generate annotations.

cd /SkillsCrafter/sim
bash instruction_generate.sh

You will now see the file train.json in data/anns folder, which can further be used for vision-action instruction tuning.

SkillsCrafter/sim
│ 
└── data
│   ├── anns
│   │   ├── sweep_to_dustpan_of_size
│   └── val
│   │   └── (10 demos preciously generated)
│   │
│   └── sweep_to_dustpan_of_size 
│       └── (demos download by link)         
│
└── instruction_generate.sh

3. SVD for Common Skill Semantic Subspaces

python data/task_name.py  # Obtain Task Instructions
python instruction_svd.py 

4. Preparing a pre-trained LLARVA model

Model	Size	Train Set	Backbone	Download
LLARVA	7B	OXE Vision-Action Instruction Pre-training Dataset	Vicuna-7B	Model

---

Instruction Tuning and Inference

1. Put the Pre-training Model

Copy the pre-trained model to the output folder, for example:

cd SkillsCrafter/sim
mkdir output
cd output
mkdir llava-lora-instruction-tuning-sweep_to_dustpan_of_size
cp -r the/path/pretrained_model  llava-lora-instruction-tuning-sweep_to_dustpan_of_size

2. Launch the instruction tuning.

cd SkillsCrafter/sim
bash task_train.sh

You will see the following structure：

SkillsCrafter/sim/task
│ 
└── task0
│   ├── task0_instruction.txt
│   └── skill_subspace.pt
│   └── task_lora.pt
│   │   
└── task1
│   ├── task1_instruction.txt
│   └── skill_subspace.pt  
│   └── task_lora.pt    
......
└── instruction_svd.py

3. Inference

To test the model on RLBench, just run

cd SkillsCrafter/sim
export SIM_ROOT=SkillsCrafter/sim
python eval.py \
rlbench.demo_path=$SIM_ROOT/data/val \
framework.eval_from_eps_number=0 \
framework.eval_episodes=25 \
rlbench.episode_length=150 \
framework.gpu=4 \
method.ckpt=path/to/llava-sweep_to_dustpan_of_size_merged # this is the download merged ckpt

---

Citation

Thanks again to all the authors at LLARVA for their contributions!
If you find our work inspiring or use our codebase in your research, please consider giving a star ⭐ and a citation.