Hierarchy Fencer

This repository is the source code for A Two-Step Approach for Physically Competitive Sports: A Case Study on Fencing, the final project for the reinforcement leaning class in NTU(CommE5069). It is built on top of Gymnasium by the Farama Foundation and branch DAC of DeepRL by Shangtong Zhang.

The environment should be installed and executed on Python 3.11.

W&B is used for logs and it will automatically create a project "Fencer".

Installation

After cloning the repository, go into the directory and use the command below to install neccessary libraries:

pip install -r requirements.txt
pip install -e .

Training

This repository currently supports training methods of SAC, TD3, A2C, PPO, and DAC (Double Actor-Critic by S. Zhang). All algorithms except for DAC are also supported by StableBaselines3.

cd gymnasium/

# Step 1

# Algorithms in StableBaselines3
python main.py <AlgorithmName> -t
# 'AlgorithmName' can be SAC, TD3, PPO, or A2C
# e.g. python main.py PPO -t

# Double Actor-Critics
python DAC.py DAC -t

# Step 2 (The two models should be trained by same method)

# Algorithms in StableBaselines3
python main.py <AlgorithmName> -t [-s2 /path/to/second/model]

# Double Actor-Critics
python DAC.py DAC -t [-s2 /path/to/second/model]

In the last command, the file extension of the path must be removed. For example, models/0000_test/DAC_100000 is used instead of models/0000_test/DAC_100000.model .

The model for the command above will be stored in gymnasium/models/[run_id]/[algorithm_name]_[steps].zip or gymnasium/models/[run_id]/[algorithm_name]_[steps].model .

Testing

# Play with itself (with first stage stationary)

# Algorithms in StableBaselines3
python main.py <AlgorithmName> -s /path/to/model

# Double Actor-Critics
python DAC.py DAC -s /path/to/model

# Play with another model (The two models should be trained by same method)

# Algorithms in StableBaselines3
python main.py <AlgorithmName> -s /path/to/model -s2 /path/to/second/model

# Double Actor-Critics
python DAC.py DAC -s /path/to/model -s2 /path/to/second/model

In the last command, the file extension of the path must also be removed.