Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Score with disk resources. #1657

Merged
merged 88 commits into from Dec 8, 2017
Merged

Conversation

@baconmania
Copy link
Contributor

@baconmania baconmania commented Nov 20, 2017

This PR rolls disk usage into Singularity's offer scoring calcs.

It also changes the default relative offer weights to 0.25 cpu; 0.65 mem; 0.10 disk based on the fact that typical apps which Singularity is intended to host are stateless, primarily I/O bound, then CPU bound, with minimal, bounded disk usage.

Still need to update tests to exercise disk resources in tests of SingularityMesosOfferScheduler#score().

/cc @ssalinas

baconmania and others added 30 commits Oct 16, 2017
defaults
Paul Schoenfelder and others added 9 commits Nov 13, 2017
S3 folders
Remove mesos dep from SingularityBase
Don't error when ArtifactManager copies duplicate files

private double longRunningUsedMemWeightForOffer = 0.70;
private double longRunningUsedMemWeightForOffer = 0.65;

This comment has been minimized.

@ssalinas

ssalinas Nov 27, 2017
Member

This actually changes the relative weights of cpu vs memory from memory being 2.333 times the weight of cpu to 2.666 times the weight of cpu. Would we instead want .27 and .63 to keep the relative weights of those two the same?

This comment has been minimized.

@ssalinas

ssalinas Nov 27, 2017
Member

Followed up in-person, will keep as is for now. mem > cpu > disk is preserved here which is the most important piece

@ssalinas
Copy link
Member

@ssalinas ssalinas commented Nov 29, 2017

FYI that cluster coordinator change should already be in master. Maybe merge master into this before getting this in staging to avoid additional merge conflicts later on?

Looks good though 👍

@tpetr
Copy link
Member

@tpetr tpetr commented Nov 29, 2017

typical apps which Singularity is intended to host are stateless, primarily I/O bound, then CPU bound, with minimal, bounded disk usage.

This is likely outside the scope of the PR, but it could be interesting if users were able to set the weights on a per-Request basis.

@baconmania
Copy link
Contributor Author

@baconmania baconmania commented Nov 29, 2017

Aye, that's a cool thought.

@ssalinas ssalinas added this to the 0.19.0 milestone Nov 30, 2017
@baconmania baconmania merged commit 02f7ac2 into report-disk-usage Dec 8, 2017
2 checks passed
2 checks passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@baconmania baconmania deleted the score-with-disk-resources branch Dec 8, 2017
@ssalinas ssalinas added hs_qa labels Feb 20, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants