Tasks queued up in Travis

[lannka]

anyway to increase the throughput?

[rsimha]

There are a few reasons for this:

• The setting for auto-cancelling multiple (duplicate) test runs for consecutive commits on one branch had been turned off. I've turned it back on, and this should de-dupe test runs from the same PRs.
• Now that we're sharding unit tests and integration tests, we're building twice, once on each shard. This will be fixed with Clean up unit tests and move all integration tests to integration/ #9404 and Optimizations to speed up Travis PR builds #9626
• We're unnecessarily doing a dist for the integration_tests shard on PRs. We only need to do so on master. This will be fixed with Don't do a dist in the integration_tests shard for PRs #9530
• Prior to sharding, a run with a presubmit error would fail early prior to the full build. Now, the unrelated shards will still be run. This is a case of a shorter wait in the average case, but a potentially longer wait in the worst case. I'm investigating a way to cancel all jobs for a run if one job fails. (Edit: It would actually be preferable not to do this, since it'd be beneficial to see results from pre_build_tests even if integration_tests were to fail, for example)

Of course, if it's feasible, adding more VMs would alleviate the worst case and greatly improve the best case.

[dreamofabear]

Addendum to second item above, we should skip building the runtime since integration tests are skipped on PR builds anyways.

[taymonbeal]

Is there any instrumentation or profiling that tells us how long is spent on each step of the CI pipeline?

[rsimha]

@taymonbeal, all the build and test tasks kicked off by pr-check.js are instrumented. Open the logs for each shard to see how long they took.

[taymonbeal]

Do you need sudo: required? Removing it means Travis doesn't have to spin up VMs. And there's no reason in principle why it shouldn't be possible to install Chrome without root, is there?

[rsimha]

/cc @erwinmombay to comment on why we needed sudo, and whether we still do.

From http://docs.travis-ci.com/user/trusty-ci-environment/, there is now beta support for trusty without sudo, although, as you point out, it doesn't spin up VMs, and therefore, doesn't offer the same kind of isolation between consecutive test runs since there might be state that persists between builds.

[taymonbeal]

It still offers full isolation. It's just that isolation is achieved through Docker containers, not through VMs, so it's faster. That's why I'm suggesting that it be used.

[rsimha]

While Docker appears to offer full isolation when it comes to persistent state (you're right), I don't believe it reserves resources like VMs do, and this may affect some of our tests. I'll spin this off into its own issue for further discussion, and if it turns out that we can do without full VMs, I'm open to making the change.

[rsimha]

Update: The PR build time (including a gulp build and a full unit test run) is now down to around 15 minutes, with a total CPU time consumption of less than 20 minutes. With this, we can close out this issue.