New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Score with disk resources. #1657

Merged
merged 88 commits into from Dec 8, 2017

Conversation

Projects
None yet
4 participants
@baconmania
Contributor

baconmania commented Nov 20, 2017

This PR rolls disk usage into Singularity's offer scoring calcs.

It also changes the default relative offer weights to 0.25 cpu; 0.65 mem; 0.10 disk based on the fact that typical apps which Singularity is intended to host are stateless, primarily I/O bound, then CPU bound, with minimal, bounded disk usage.

Still need to update tests to exercise disk resources in tests of SingularityMesosOfferScheduler#score().

/cc @ssalinas

baconmania and others added some commits Oct 16, 2017

catch nulls
defaults

Paul Schoenfelder and others added some commits Nov 13, 2017

Merge pull request #1648 from HubSpot/base_no_mesos
Remove mesos dep from SingularityBase
Merge pull request #1649 from HubSpot/s3_dupe_file_dl
Don't error when ArtifactManager copies duplicate files
private double longRunningUsedMemWeightForOffer = 0.70;
private double longRunningUsedMemWeightForOffer = 0.65;

This comment has been minimized.

@ssalinas

ssalinas Nov 27, 2017

Member

This actually changes the relative weights of cpu vs memory from memory being 2.333 times the weight of cpu to 2.666 times the weight of cpu. Would we instead want .27 and .63 to keep the relative weights of those two the same?

This comment has been minimized.

@ssalinas

ssalinas Nov 27, 2017

Member

Followed up in-person, will keep as is for now. mem > cpu > disk is preserved here which is the most important piece

@ssalinas

This comment has been minimized.

Member

ssalinas commented Nov 29, 2017

FYI that cluster coordinator change should already be in master. Maybe merge master into this before getting this in staging to avoid additional merge conflicts later on?

Looks good though 👍

@tpetr

This comment has been minimized.

Member

tpetr commented Nov 29, 2017

typical apps which Singularity is intended to host are stateless, primarily I/O bound, then CPU bound, with minimal, bounded disk usage.

This is likely outside the scope of the PR, but it could be interesting if users were able to set the weights on a per-Request basis.

@baconmania

This comment has been minimized.

Contributor

baconmania commented Nov 29, 2017

Aye, that's a cool thought.

@ssalinas ssalinas added this to the 0.19.0 milestone Nov 30, 2017

@baconmania baconmania merged commit 02f7ac2 into report-disk-usage Dec 8, 2017

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@baconmania baconmania deleted the score-with-disk-resources branch Dec 8, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment