You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I missed a Windows test failure in CL 220645 because I forgot to run it against the windows-amd64-longtest SlowBot. I forgot to run it against that SlowBot because I'm not in the habit of doing so.
I'm not in the habit of running that SlowBot because it is currently much too slow. To pick some relevant runs:
The first run on CL 220717 started at 5:07 PM and completed at 5:31 PM (24 minutes).
The second run on that CL started at 5:51 PM and completed at 6:33 PM (42 minutes).
The run on CL 220722 started at 5:48 PM and completed at 6:28 PM (40 minutes).
In contrast, a regular TryBot typically caps out around 10 minutes (#32632), and we consider runs that take longer than 20 minutes to be unacceptably slow (#36629, #36482).
Since there is nothing particularly special about the hardware needed to run the longtest builds (they're just large VMs), I think we should adjust the builder configuration to run the -longtest SlowBots with 4 or more shards each. That way, the end-to-end latency impact of adding one of these bots to a CL will be minimal, and we will not only have less of a disincentive to using them, but also have much faster feedback in order to inform revert-or-fix decisions when one breaks.
The text was updated successfully, but these errors were encountered:
The longtest builders are currently primarily post-submit builders,
where it's okay for them to be as slow as they need to be in order
to provide additional test coverage. In this context, whether they
take 40 minutes or 50 makes little difference.
The longtest builders are also sometimes requested via SlowBots for
changes that are riskier than usual, or otherwise desire additional
coverage beyond the normal TryBots. They're also always enabled for
CLs to release branches. In such contexts, speeding up SlowBot runs
from 40 minutes to 20 or less would be appreciated and in turn help
people use longtest SlowBots more frequently.
Longtest builders are already configured to use sharded tests.
Configure them to use additional helpers to speed up test execution.
Try out 3, 5, and 9 helpers to see how much it helps before settling.
Trust: Dmitri Shuralyov <email@example.com>
Run-TryBot: Dmitri Shuralyov <firstname.lastname@example.org>
TryBot-Result: Go Bot <email@example.com>
Reviewed-by: Alexander Rakoczy <firstname.lastname@example.org>
Reviewed-by: Carlos Amedee <email@example.com>