Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: shard and scale the `longtest` SlowBots #37439

Open
bcmills opened this issue Feb 25, 2020 · 1 comment
Open

x/build: shard and scale the `longtest` SlowBots #37439

bcmills opened this issue Feb 25, 2020 · 1 comment

Comments

@bcmills
Copy link
Member

@bcmills bcmills commented Feb 25, 2020

I missed a Windows test failure in CL 220645 because I forgot to run it against the windows-amd64-longtest SlowBot. I forgot to run it against that SlowBot because I'm not in the habit of doing so.

I'm not in the habit of running that SlowBot because it is currently much too slow. To pick some relevant runs:

  • The first run on CL 220717 started at 5:07 PM and completed at 5:31 PM (24 minutes).
  • The second run on that CL started at 5:51 PM and completed at 6:33 PM (42 minutes).
  • The run on CL 220722 started at 5:48 PM and completed at 6:28 PM (40 minutes).

In contrast, a regular TryBot typically caps out around 10 minutes (#32632), and we consider runs that take longer than 20 minutes to be unacceptably slow (#36629, #36482).

Since there is nothing particularly special about the hardware needed to run the longtest builds (they're just large VMs), I think we should adjust the builder configuration to run the -longtest SlowBots with 4 or more shards each. That way, the end-to-end latency impact of adding one of these bots to a CL will be minimal, and we will not only have less of a disincentive to using them, but also have much faster feedback in order to inform revert-or-fix decisions when one breaks.

CC @golang/osp-team

@cagedmantis
Copy link
Contributor

@cagedmantis cagedmantis commented Feb 28, 2020

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.