Skip to content
This repository has been archived by the owner on Apr 24, 2023. It is now read-only.

Thread compute cluster through Fenzo #1136

Merged
merged 4 commits into from May 13, 2019

Conversation

scrosby
Copy link
Member

@scrosby scrosby commented May 7, 2019

Changes proposed in this PR

  • Thread the compute-cluster name through Fenzo from the offer, instead of getting it via the global.

Why are we making these changes?

This ensures that the compute cluster is attached to the offer, an enabling refactor for supporting more than one cluster. Now we know to whom to send the response.

@scrosby scrosby requested a review from pschorf May 7, 2019 13:57
@scrosby scrosby changed the title Outgoing/compute cluster fenzo Thead compute cluster through Fenzo May 7, 2019
(defn get-default-cluster-name-for-legacy
"What cluster name to put on for legacy jobs when generating their compute-cluster."
[]
@default-cluster-name-for-legacy)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we get rid of the atom and just have this read the config? This shouldn't change over the lifetime of the process.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I'll make a parallel change in the new patch. :)

; A hack to store the mesos cluster name, until we refactor the code so that we support multiple clusters. In the long term future
; this is probably replaced with a function from driver->cluster-id, or the cluster name is propagated by function arguments and
; closed over.
(def mesos-cluster-name-hack (atom nil))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this in addition to get-default-cluster-name-for-legacy?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mesos-cluster-name-hack is a hack that will go away when we restructure the initialization. get-default-cluster-for-legacy will exist forever in order to back-patch a cluster into legacy task structures.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I buy that. Same comment as above, though: can we replace the atom with a string read from config?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok.

scheduler/src/cook/mesos/api.clj Show resolved Hide resolved
@DaoWen DaoWen changed the title Thead compute cluster through Fenzo Thread compute cluster through Fenzo May 10, 2019
@pschorf pschorf merged commit 3552068 into twosigma:master May 13, 2019
@scrosby scrosby deleted the outgoing/compute-cluster-fenzo branch May 30, 2019 21:26
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants