Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hydra multirun causes Segmentation fault (core dumped) #96

Open
generush opened this issue Dec 8, 2022 · 0 comments
Open

Hydra multirun causes Segmentation fault (core dumped) #96

generush opened this issue Dec 8, 2022 · 0 comments

Comments

@generush
Copy link

generush commented Dec 8, 2022

Hello,

Thank you for developing this repo and open sourcing it, it has been very helpful for my own research work.

I recently tried to utilize the Hydra multirun command to conduct a parameter sweep of my deep RL network. However, upon doing that, I received a seg fault, and I am not sure how to resolve it. I hope we can find a solution! If there are more convenient methods of running parameters sweeps instead of using Hydra, I would also be curious to hear your thoughts.

Segmentation fault (core dumped)

The seg fault occurs after calling isaacgymenvs.make and entering _create_env. Typically, the seg fault occurs 50-75% of the way through _create_env, though it seems stochastic.

I continued to investigate this problem, and wanted to know if it was specific to my code or not. So I went to the IsaacGymEnvs repo, and inside the folder IsaacGymEnvs/isaacgymenvs, ran via the terminal:

python train.py -m seed=0,1 max_iterations=10 force_render=False

(this conducts two runs, seed=0 and seed=1, I set the other parameters so it would train the first run within seconds and move quikcly to the second run) and received the same seg fault, this time with an error message:

[Error] [carb.gym.plugin] Function GymGetActorDofStates cannot be used with the GPU pipeline after simulation starts.  Please use the tensor API if possible.  See docs/programming/tensors.html for more info.
Segmentation fault (core dumped)

At this point, I realized it may be a problem within the IsaacGymEnvs code base, and that it's unlikely I could resolve this problem on my own, so I decided to communicate it here. If there is some glaring mistake or misunderstanding I am making, I welcome feedback! Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant