Is darwin-amd64 the only trybot that can't scale with demand?
Currently, yes. There was also linux-arm in the trybot set, which had a fixed number (50), but those were disabled due to #22748 (first) and #22749 (most recently). I could probably re-enabled them, and wait until their networking sucks again, or redesign things to not depend on the network as much, which is unfortunate.
We could buy more Macs, or hope that #19178 and #23858 are sufficient.
I'm not sure this is the only problem, though. When I was trybot'ing those changes earlier today the dashboard was pretty quiet and I don't think any trybots other than mine were running, but it still took over two hours for all of the darwin-amd64 trybots to finish. It's only 16 CLs, so shouldn't 20 Mac VMs be enough to handle this plus a bit?
@aclements, well, each CL consumes 3 Mac VMs for sharding (or up to 4 for Trybots). And some of the VMs are currently statically partitioned into distinct roles (some for macOS 10.8, some for macOS 10.12 Sierra, etc).
We expect only 15 VMs for macOS 10.11, which is what we run for TryBots.