[Question] rl_zoo3 optimization pipeline for ros-based custom env

[ammar-n-abbas]

❓ Question

Hello,

I am trying to perform hyperparameter tuning on a ROS-based custom env. It works fine for training and loading the trained agent, however, for optimization case I get some communication errors with ROS services and parameter space.

Does anyone have an idea of possible problem related to this?

cmd:
python -m rl_zoo3.train --algo tqc --env UR5PickAndPlaceEnv-v0 -optimize

Thanks

Checklist

• I have checked that there is no similar issue in the repo
• I have read the SB3 documentation
• I have read the RL Zoo documentation
• If code there is, it is minimal and working
• If code there is, it is formatted using the markdown code blocks for both code and stack traces.

[ammar-n-abbas]

I found a possible problem:

In the training pipeline at setup_experiment method why is

env.close()

called when the optimization param is True?

When I uncomment it, it runs, however, does not optimize.

if self.continue_training:  
     model = self._load_pretrained_agent(self._hyperparams, env)  
elif self.optimize_hyperparameters:  
    env.close()  
    return None  

[ammar-n-abbas]

According to the openai_ros interface, when env.close() is invoked, it also shutdowns the ros signals and calls

rospy.signal_shutdown

Therefore, it drops the communication