This is the original code for the ICLR2023 submission Effective Offline Reinforcement Learning via Conservative State Value Estimation. We largely follow the implementaion in d3rlpy.
Check Install form source in Installation of d3rlpy
To reproduce any experiments with our proposed algorithm, you should first train the dynamic model and then train the offline rl models.
- Learn and store the dynamic model. You should run
python dynamics.py
in d3rlpy/reproductions/offline directory to get the trained dynamics models.
- Train and evaluate with our algorithm. After replacing the directories of dynamics models, you can start trainning through
python csve_final_adroit.py
in d3rlpy/reproductions/offline directory.
You should adjust the hyperparameters in the arguments to get the results in our paper.
The detailed results will be stored in the logs_09 directory. One can also view the results in directory exp_09 using tensorboard.
For any questions, you can post them in the openreview thread under our submission.