Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for separate external address for SpecCluster scheduler #2963

Merged
merged 2 commits into from Aug 17, 2019

Conversation

@jacobtomlinson
Copy link
Member

commented Aug 17, 2019

Minimal implementation which closes #2962.

Allow SpecCluster scheduler tasks to set an optional external_address property which will be used for the RPC connection instead of the scheduler address.

Allows clusters to be created and managed externally while allowing workers to continue to talk on an internal network.

@mrocklin

This comment has been minimized.

Copy link
Member

commented Aug 17, 2019

If we do this then we might also want to override the dashboard_link property from the Cluster superclass.

@property
def dashboard_link(self):
try:
port = self.scheduler_info["services"]["dashboard"]
except KeyError:
return ""
else:
host = self.scheduler_address.split("://")[1].split(":")[0]
return format_dashboard_link(host, port)

@mrocklin

This comment has been minimized.

Copy link
Member

commented Aug 17, 2019

Ah, never mind about the dashboard_link, that uses scheduler_address, which uses scheduler_comm.address

@mrocklin

This comment has been minimized.

Copy link
Member

commented Aug 17, 2019

OK, I'm +1 on this. If you're ok with things I'll merge on passed tests.

@mrocklin mrocklin merged commit 31e7754 into dask:master Aug 17, 2019

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.