Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

applying FIFO in Concurrent trials through Ray Tune #20

Open
Ne-oL opened this issue Aug 3, 2022 · 0 comments
Open

applying FIFO in Concurrent trials through Ray Tune #20

Ne-oL opened this issue Aug 3, 2022 · 0 comments

Comments

@Ne-oL
Copy link

Ne-oL commented Aug 3, 2022

the max_concurrent feature is extremely helpful to speed up the optimization process though parallelization. However, the problem is that to start the next set of trials, it has to wait until ALL the concurrent trials (defined by max_concurrent) finishes. which is counter intuitive in my use case as some combination of hyperparameters takes too long to finish compared to other combinations. which leaves most of the CPUs Idle most of the time due to one trial taking too long. does HEBO need to have all concurrent running trials finish to suggest the next combination or is there a way to make it suggest the next combination once any of the concurrent running trails finish so no CPUs stays Idle until all of them finish?
btw, I'm using Ray Tune to deploy HEBO.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant