-
Notifications
You must be signed in to change notification settings - Fork 107
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PBT with parallel simulations #268
Comments
Hi! Technically it should work, except it won't be possible to have multiple agents controlled by different policies in the same environment. IsaacGym works in batched mode, which keeps everything on the GPU in big batches. For PBT to work you'd need to set num_policies > 1 and num_workers = num_policies (each worker will run an env controlled by a corresponding policy). There must be a limit of how many IsaacGym processes you can run on a single machine. On a multi-GPU machine you can probably run a lot of them. We also have a decentralised PBT implementation in the works that can run on multiple machines, it will be merged at some point, but no ETA on this yet, it's stuck in some corporate reviews. |
Thanks for the suggestion. I tried creating multiple instances of Isaac Gym using multi workers, but it throws an error:
|
Are you running in serial mode? In this case everything runs in the same process and you will have conflicts. |
If having multiple IsaacGym processes is not an option, I can also suggest writing a wrapper which starts an IG instance in a separate process and splits the vector of agents into sub-vectors which are each going to be controlled by a different policy. These sub-vector envs can run on separate workers thus making Sample Factory happy. Hope it makes sense. It's a bit of work but definitely can be done. |
I can only run a maximum of 2 IsaacGym processes on my single GPU machine. Running with |
Hi,
Does PBT work with the Isaac gym envs?
Thanks
The text was updated successfully, but these errors were encountered: