Maintenance: Sharding and parallelizing RSpec tests in CI

[kevinrobinson]

This started as experimental, but going to ship it now 🚢

This PR cuts total build time to ~4min! The only downside was that the way the CoverageChecker worked each shard would fail, because none of the partial test runs reach 100% coverage on the expected files. So I updated this to upload the coverage artifacts, and then added another step to merge them and run the coverage check.

This ran into a few issues:

• SimpleCov's collate function seemed to work but the methods that it normally provides had missing data (eg, filenames). So this still runs SimpleCov on each test run to produce artifacts, but uses the plain Ruby Coverage output it produces for merging the shards on its own in plain Ruby. This also lets the coverage enforcer step us just plain Ruby rather than all the gems etc.
• The GitHub download action didn't support multiple files, but the v2-preview does. That has a bug with not properly unzipping artifacts though, so we work around that in here, with comments to the upstream issue which should be resolved soon.
• Unrelated: there were some severe latency spikes with cached folders for Ruby gems. I commented upstream and it hasn't come up again this afternoon.
• Unrelated: there's an incident now impacting node download servers, so this works around and adds comments on that.
• Unrelated: Tuned down the Postgres health check, it was waiting ~10 seconds longer than needed.

EDIT2: Verifying code coverage check works, link:

EDIT3: example:

EDIT4: 🎉

[kevinrobinson]

This works great, and would cut total build time to ~4min! Seems worth doing because of that; the only downside is that the way the CoverageChecker works now each shard fails, because none of the partial test runs reach 100% coverage on the expected files. We could update that or tackle it another way, and the 2x speedup cutting off over 5min on the build time seems worth it, so will consider.

[kevinrobinson]

selfie