TactileRL

TactileRL is a reinforcement learning project for contact-rich robotic manipulation in Isaac Lab. The repository contains a custom PPO training pipeline, Isaac Lab environments, and tactile manipulation tasks for learning behaviors such as non-prehensile cube rotation, lifting, reaching, and shape-aware manipulation.

The main experimental code lives in bifranka-RL/, while the repo also includes an IsaacLab/ directory for the underlying Isaac Lab framework and tooling.

Repository Structure

TactileRL/
├── IsaacLab/          # Isaac Lab framework dependency/source
└── bifranka-RL/       # Custom PPO + tactile RL task code
    ├── assets/        # Robot/object assets
    ├── tasks/         # Custom RL environments and reward variants
    ├── ppo.py         # PPO training script
    ├── eval.py        # Policy evaluation script
    ├── play.py        # Playback / interactive script
    └── README.md

Key Features

Custom PPO implementation for Isaac Lab environments Massive parallel simulation support through --num-envs Task-specific reward engineering for tactile manipulation Environments for reaching, lifting, pick-place-lift, Allegro hand control, and non-prehensile manipulation Shape discriminator task implementation for tactile object/shape reasoning TensorBoard and optional Weights & Biases logging Model checkpoint saving and evaluation utilities

Tasks

The bifranka-RL/tasks/ directory contains multiple environment variants, including:

• nonprehensile.py — cube/object rotation and non-prehensile manipulation
• allegro.py, allegro_2.py, allegro_3.py — Allegro hand manipulation environments
• allegro_shape_discriminator.py — shape discriminator based tactile task
• lift*.py — lifting tasks with different control/reward formulations
• reach*.py — reaching tasks with joint-space and Cartesian control variants
• pick_place_lift.py — pick, place, and lift manipulation task

Training

From the bifranka-RL/ directory:

python ppo.py \
  --env-id nonprehensile \
  --num-envs 16384 \
  --track \
  --save-model \
  --exp-name test \
  --headless

This trains a PPO policy on the selected task. The default environment is nonprehensile, and environments are dynamically loaded from tasks/.py. Training logs and checkpoints are saved under: bifranka-RL/runs/<experiment_name>_/

Evaluation

To evaluate a saved run:

python eval.py \
  --env-id nonprehensile \
  --num-envs 1 \
  --run test_20250819_200015 \
  --livestream 1

Replace test_20250819_200015 with the run folder you want to evaluate.

PPO Details

The PPO trainer supports:

Generalized Advantage Estimation Learning-rate annealing Clipped policy objective Clipped value loss Entropy regularization Gradient clipping TensorBoard logging Optional WandB tracking Periodic model checkpointing

Important arguments include:

--env-id              Task name from bifranka-RL/tasks/
--num-envs            Number of parallel environments
--num-steps           PPO rollout length
--learning-rate       Optimizer learning rate
--gamma               Discount factor
--gae-lambda          GAE lambda
--num-minibatches     PPO minibatches
--update-epochs       PPO update epochs
--track               Enable WandB logging
--save-model          Save checkpoints
--headless            Run Isaac Lab without GUI

Project Motivation

Robotic manipulation often fails when vision alone is insufficient, especially during contact-rich interactions where objects slip, rotate, or become occluded. Tactile feedback provides direct information about contact, object geometry, and interaction forces.

This project explores how reinforcement learning policies can use tactile/contact information to learn manipulation behaviors such as rotating a cube, lifting objects, and distinguishing object shapes through touch.

Example Workflow

git clone https://github.com/ShrishChou/TactileRL.git
cd TactileRL/bifranka-RL

python ppo.py --env-id nonprehensile --num-envs 16384 --track --save-model --exp-name cube_rotation --headless

python eval.py --env-id nonprehensile --num-envs 1 --run cube_rotation_<timestamp> --livestream 1

Notes

This repository is experimental research code. Many task files represent different reward functions, control modes, and environment iterations used while developing tactile RL behaviors.

Author

Shrish Choudhary