Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parallel building #141

Open
notriddle opened this Issue Apr 6, 2017 · 2 comments

Comments

Projects
None yet
2 participants
@notriddle
Copy link
Member

notriddle commented Apr 6, 2017

Instead of batching stuff together, run every item in the backlog immediately with everything ahead of it merged in. This has a couple nice side-effects:

  • It reduces the time taken from O(E*log(N)) to O(E).
  • It results in a history that doesn't have arbitrary octopus merges.

On the other hand...

  • It increases the amount of builds run from O(E*log(N)) to O(E*N)
  • The time decrease only applies if you have an arbitrarily high number of build nodes. If you run out of nodes, it jumps to O(N) again.

So we do want to provide both.

@Macarse

This comment has been minimized.

Copy link
Contributor

Macarse commented Apr 6, 2017

Here's a nice diagram explanation of the algorithm: https://docs.openstack.org/infra/zuul/gating.html#testing-in-parallel

@notriddle

This comment has been minimized.

Copy link
Member Author

notriddle commented May 31, 2017

I should probably mention, just for completeness's sake, that this is going to be awhile.

Making good on parallel building requires near-bottomless build capacity and the ability for bors to insta-cancel builds. Travis limits free projects to something like five concurrent builds, which would be enough for bors-ng/bors-ng itself, but isn't going to cut it for rust-lang/rust.

Maybe we could use Travis's "auto-cancel" feature to drop known-bad builds from the queue? That could work, but I'd rather have something where bors can proactively throw errors if it's misconfigured. Being easy to set up matters.

@notriddle notriddle added the I-perf label Jul 28, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.