Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Thread compute cluster through Fenzo #1136

Merged
merged 4 commits into from May 13, 2019

Conversation

Projects
None yet
2 participants
@scrosby
Copy link
Member

commented May 7, 2019

Changes proposed in this PR

  • Thread the compute-cluster name through Fenzo from the offer, instead of getting it via the global.

Why are we making these changes?

This ensures that the compute cluster is attached to the offer, an enabling refactor for supporting more than one cluster. Now we know to whom to send the response.

@scrosby scrosby requested a review from pschorf May 7, 2019

@scrosby scrosby changed the title Outgoing/compute cluster fenzo Thead compute cluster through Fenzo May 7, 2019

(defn get-default-cluster-name-for-legacy
"What cluster name to put on for legacy jobs when generating their compute-cluster."
[]
@default-cluster-name-for-legacy)

This comment has been minimized.

Copy link
@pschorf

pschorf May 9, 2019

Contributor

Can we get rid of the atom and just have this read the config? This shouldn't change over the lifetime of the process.

This comment has been minimized.

Copy link
@scrosby

scrosby May 10, 2019

Author Member

Done. I'll make a parallel change in the new patch. :)

; A hack to store the mesos cluster name, until we refactor the code so that we support multiple clusters. In the long term future
; this is probably replaced with a function from driver->cluster-id, or the cluster name is propagated by function arguments and
; closed over.
(def mesos-cluster-name-hack (atom nil))

This comment has been minimized.

Copy link
@pschorf

pschorf May 9, 2019

Contributor

Why do we need this in addition to get-default-cluster-name-for-legacy?

This comment has been minimized.

Copy link
@scrosby

scrosby May 10, 2019

Author Member

mesos-cluster-name-hack is a hack that will go away when we restructure the initialization. get-default-cluster-for-legacy will exist forever in order to back-patch a cluster into legacy task structures.

This comment has been minimized.

Copy link
@pschorf

pschorf May 10, 2019

Contributor

Ok, I buy that. Same comment as above, though: can we replace the atom with a string read from config?

This comment has been minimized.

Copy link
@scrosby

scrosby May 10, 2019

Author Member

Ok.

Show resolved Hide resolved scheduler/src/cook/mesos/api.clj

@DaoWen DaoWen changed the title Thead compute cluster through Fenzo Thread compute cluster through Fenzo May 10, 2019

Scott Crosby added some commits May 10, 2019

Scott Crosby

@pschorf pschorf merged commit 3552068 into twosigma:master May 13, 2019

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.