New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WIP: worker environments #505
Conversation
- Returns set of workers that fit the names/hosts/environments provided - Now works with environment names as well - Used most places where `self.coerce_address` was used before.
workers.extend([host + ':' + port | ||
for port in self.host_info[host]['ports']]) | ||
# TODO replace with worker_list | ||
workers = self.workers_set(workers) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hooray!
@@ -1091,6 +1092,28 @@ def get_dataset(self, name): | |||
return sync(self.loop, self._get_dataset, tokey(name)) | |||
|
|||
@gen.coroutine | |||
def _register_environment(self, name, env): | |||
if not isinstance(env, Environment): | |||
env = Environment(condition=env) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we could also do this with keyword arguments, like the following:
def register_environment(self, name, environment=None, condition=None):
...
Considering names for a moment. I'm colliding a bit here with the recent conda work that allows the client to upload conda environments for all of the workers to restart themselves in. This feature is unpublished and so can be easily renamed, but the current name is Thoughts on names for either of the two concepts currently called environments? |
This PR has gone stale. Closing for now. |
Todo:
Client.register_environment
updates state on scheduler and workersClient.submit
can take environment names inworkers=