While doing some test work on another PR, I accidentally ran a
Meteor's CI infrastructure is configured to exclude certain
This commit adds a new
Meteor's CI infrastructure is configured to exclude certain `self-test`'s on each run. These excludes are specified in each CI environment's config file, and included when running `meteor self-test`. Developers running `meteor self-test` locally however are not using these excludes by default, so developer's have to manually look up the current exclude list from one of the CI configs, then add these excludes to their own `meteor self-test` call manually. This commit adds a new `skip` option to Meteor's `self-test` system, that can be used to skip adding/running a defined `self-test` (similar in concept to Mocha's `skip` feature). This provides a way to skip the running of older `self-test`'s that are no longer needed, but allows them to be preserved in the `self-test` suite, for future reference. With this functionality in place, and the older test suites updated to use it, Meteor's base CI excludes no longer need to be maintained in their respective config files. The excludes are all managed at the source (the test definition), and can be leveraged by anyone/anything calling `meteor self-test`.
@benjamn I've pushed some tweaks to make this fall more in-line with the existing self-test summary. How about this:
So we don't show the name of the manually ignored tests, but we do list a manually ignored count (and mention that they were ignored by selftest.skip). This lines up with the approach used for the other skip tags, like "non-matching" - it doesn't show the names of the non-matching tests, just a count.
Let me know if this works - thanks!