New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove "local execution delay" from dynamic spawn scheduler #7327

Open
jmmv opened this Issue Feb 1, 2019 · 1 comment

Comments

Projects
None yet
2 participants
@jmmv
Copy link
Contributor

jmmv commented Feb 1, 2019

The dynamic spawn scheduler implements a "local execution delay", controlled via the --experimental_local_execution_delay flag, that we should remove.

This delay exists so that, in the case of a build that is fully cached remotely, we don't end up running any actions locally. The delay allows the scheduler to try to score a cache hit for every action and, assuming all cache hits succeed before this delay, the local counterparts never start. But as soon as an action takes longer than this delay, the scheduler assumes that the build is unclean and it will have to do work, so it starts also scheduling actions locally.

This delay is only paid once per build and, because it's set to 1 second by default, it's not a big penalty -- but regardless, it's ugly and we should remove it.

To remove this delay, we can propagate a cache miss signal from the remote execution logic to the dynamic scheduler. We can then use this signal to decide when it's time for the dynamic spawn scheduler to start running actions locally.

@dslomov

This comment has been minimized.

Copy link
Contributor

dslomov commented Feb 15, 2019

(please do not assign issues to more than one team-*)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment