Make sure subprocess/process/thread/dask can create a runner instance #816
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Delaunay
force-pushed
the
issue_807
branch
7 times, most recently
from
d3dbfe7
to
29ea940
Compare
|
@jbeirer I think I was able to reproduce your issue (#807) Could you provide a minimal example that reproduces the issue ? New ExceptionOriginal Issue |
|
Hi @Delaunay, thanks for looking into this. Indeed, I did not have these kind of issues in the previous 0.2.1 version. The most minimal reproducer I can come up with in order to reproduce #807 is running two experiments in parallel: # Orion includes
from orion.executor.base import executor_factory
from distributed import Client
from orion.client import build_experiment
# Dask includes
import dask
import dask.distributed
# Dummy method
def main():
return [ {"name": "error", "type": "objective", "value": 1}]
if __name__ == '__main__':
########################################################################
### Assumes cluster, space, storage and algorithm variables are set ###
########################################################################
# Set number of trials and workers
nMaxTrials = 1
nWorkers = 1
# Initialize client
client = Client(cluster)
cluster.scale(nWorkers)
# Create DASK executor
executor = executor_factory.create('dask', n_workers = nWorkers, client = client)
# Encapsulate experiment execution in method to parellelise experiments
def exec_experiment(executor, nWorkers, experiment):
experiment.workon(main, n_workers = nWorkers)
# Build two experiments to run in parallel
experimentNames = ['experiment1', 'experiment2']
exp_futures = []
for experimentName in experimentNames:
experiment = build_experiment(name = name, max_trials = nMaxTrials, space = space, storage = storage, algorithms = algorithm, executor = executor)
future = client.submit(exec_experiment, executor, nWorkers, experiment)
exp_futures.append(future)
# Gather results
for exp_future in exp_futures:
client.gather(exp_future)Hope this helps! Cheers, Joshua |
|
@jbeirer I think I identified the root cause, the issue should be resolved now |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi there! Thank you for contributing. Feel free to use this pull request template; It helps us reviewing your work at its true value.
Please remove the instructions in italics before posting the pull request :).
Description
Describe what is the purpose of this pull request and why it should be integrated to the repository.
When your changes modify the current behavior, explain why your solution is better.
If it solves a GitHub issue, be sure to link it.
Changes
Give an overview of the suggested solution.
Checklist
This is simply a reminder of what we are going to look for before merging your code.
Add an
xin the boxes that apply.If you're unsure about any of them, don't hesitate to ask. We're here to help!
You can also fill these out after creating the PR if it's a work in progress (be sure to publish the PR as a draft then)
Tests
$ tox -e py38; replace38by your Python version if necessary)Documentation
Quality
$ tox -e lint)Further comments
Please include any additional information or comment that you feel will be helpful to the review of this pull request.