Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

introduce offer cache to allow better resource allocation #1402

Merged
merged 19 commits into from Mar 10, 2017
Merged

Conversation

@wsorenson
Copy link
Contributor

@wsorenson wsorenson commented Jan 19, 2017

Currently, does not use cached offers.

@ssalinas ssalinas modified the milestone: 0.14.0 Jan 20, 2017
@wsorenson wsorenson added the hs_qa label Jan 26, 2017
@ssalinas

This comment has been minimized.

just curious, is there a performance improvement from this?

This comment has been minimized.

Copy link
Member

@jhaber jhaber replied Feb 1, 2017

Internally the logger will do this same check so I think it should be about the same

This comment has been minimized.

Copy link
Contributor Author

@wsorenson wsorenson replied Feb 4, 2017

Not in this case - only when you can skip method evaluations. I suppose in this case toString() is probably called after the logger decides to log, so its irrelevant

wsorenson added 6 commits Feb 1, 2017

private void checkOut() {
LOG.trace("Checking out offer {}", offerId);
Preconditions.checkState(offerState == OfferState.AVAILABLE, "Offer %s was in state %s", offerId, offerState);

This comment has been minimized.

@ssalinas

ssalinas Feb 3, 2017
Member

Saw a sentry come through for this the other day. This exception will bubble all the way back up to the LeaderOnlyPoller and cause us to abort. Should we maybe catch this and just skip the offer in the case that something else has it checked out?

wsorenson added 2 commits Feb 3, 2017
@wsorenson

This comment has been minimized.

Copy link
Contributor Author

@wsorenson wsorenson commented on e4c42f1 Feb 4, 2017

In my test case of 1000 tasks and 250 offers, saw a total improvement of about 3m21s to 14s

@ssalinas ssalinas added the hs_stable label Feb 24, 2017

public void returnOffer(CachedOffer cachedOffer);

public List<Offer> peakOffers();

This comment has been minimized.

@tpetr

tpetr Mar 2, 2017
Member

peekOffers()?

ssalinas added 2 commits Mar 10, 2017
@ssalinas
Copy link
Member

@ssalinas ssalinas commented Mar 10, 2017

Fixed the merge conflicts and re-tested in each environment as well, going to merge this one 👍

@ssalinas ssalinas merged commit 118c4e9 into master Mar 10, 2017
0 of 2 checks passed
0 of 2 checks passed
continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
continuous-integration/travis-ci/push The Travis CI build is in progress
Details
@ssalinas ssalinas removed hs_qa labels Mar 10, 2017
@miry
Copy link

@miry miry commented May 8, 2017

@wsorenson how to disable this this feature?

I could not find related info in http://getsingularity.com/Docs/reference/configuration.html.

I think this feature eats all Mesos resources and then free some of them in 1 minute, and then again takes. So other frameworks could not run job fast.

@tpetr
Copy link
Member

@tpetr tpetr commented May 8, 2017

@miry It looks like the documentation needs to be updated, thanks for the heads up. You can disable offer caching by setting cacheOffers: false in your Singularity config YAML.

@ssalinas @wsorenson thoughts on having offer caching disabled by default?

@ssalinas
Copy link
Member

@ssalinas ssalinas commented May 8, 2017

For the sake of those running in multi-framework clusters it might make sense to have it off by default. I can submit a PR with that and the documentation update

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants
You can’t perform that action at this time.