Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 

Curious: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning

Implementation of CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning.

This implementation is based on the OpenAI baseline implementation of Hindisght Experience Replay and Deep Deterministic Policy Gradient (included in this repo).

This implementation requires the installation of the gym_flowers module, which overrides gym to enable the use of custom environments such as the one we use in this paper (Modular Multi-Goal Fetch Arm).

The video of the results can be seen here

To run an experiment, run:

python3 /curious/baselines/her/experiment/train.py

options include:

  • --num_cpu: Number of cpus. The paper uses 19 cpus (as in the original paper presenting this HER implementation. Running the code with fewer cpus for a longer time is NOT equivalent.
  • --env: string of the gym_flowers env. Possible choices are MultiTaskFetchArm4-v5 (4 tasks: Reach, Push, Pick and Place, Stack), MultiTaskFetchArm8-v5 (same with 4 distracting tasks).
  • --task_selection: use 'active_competence_progress' to use learning progress to guide module selection, 'random' otherwise.
  • --goal_selection: 'random' is the only supported here.
  • --goal_replay: 'her' uses Hindisght Experience Replay or 'none'.
  • --task_replay: 'replay_task_cp_buffer' uses learning progress to sample into module-relevant replay buffers. 'replay_task_random_buffer' samples into a buffer associated to a random module.
  • --structure: 'curious' uses the curious algorithm, 'task_experts' uses one UVFA policy per module.
  • --trial_id: trial identifier.

Results are saved in: /curious/baselines/her/experiment/save/env_name/trial_id/

About

Implementation of CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning

Resources

License

Releases

No releases published

Packages

No packages published