Skip to content

Don't allocate more executors to a build than total subjobs#284

Merged
tjlee0909 merged 1 commit intomasterfrom
done_allocate_more_executors_to_build_than_subjobs
Jan 5, 2016
Merged

Don't allocate more executors to a build than total subjobs#284
tjlee0909 merged 1 commit intomasterfrom
done_allocate_more_executors_to_build_than_subjobs

Conversation

@tjlee0909
Copy link
Copy Markdown

This addresses #273

@boxcla
Copy link
Copy Markdown

boxcla commented Jan 1, 2016

Verified that @tjlee0909 has signed the CLA. Thanks for the pull request!

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought that breaking up the conditionals was much easier to read than all of the 'ands' and 'nots' chained together.

@tjlee0909 tjlee0909 force-pushed the done_allocate_more_executors_to_build_than_subjobs branch from 958d9d2 to 1e27531 Compare January 1, 2016 01:50
@tjlee0909
Copy link
Copy Markdown
Author

I posted a comment about this in #273 , but to reiterate:

I'm going for low-hanging fruit here with the somewhat common manifestation of this bug where the max_executors is set to much higher than the actual number of subjobs calculated by the atomizer.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self: maybe should compare to self._build._unstarted_subjobs.qsize() instead.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I actually prefer checking the total number of subjobs. What were you thinking would be the advantage of checking _unstarted_subjobs.qsize()?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the scenario where the slaves being allocated are trickling in (being released from other builds onto this one), then the length of all subjobs is an overestimate, whereas the unstarted_subjobs.qsize() is more of an exact fit.

It's really not a big deal though. I'm happy with leaving it as is and merging it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah agreed / not a big deal. 🤘

@josephharrington
Copy link
Copy Markdown
Contributor

Hallelujah! 👍

tjlee0909 pushed a commit that referenced this pull request Jan 5, 2016
…d_than_subjobs

Don't allocate more executors to a build than total subjobs
@tjlee0909 tjlee0909 merged commit 21837e1 into master Jan 5, 2016
@tjlee0909 tjlee0909 deleted the done_allocate_more_executors_to_build_than_subjobs branch April 4, 2016 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants