Update JS testing guide to include Jest and info on generating coverage by vividviolet · Pull Request #267 · Shopify/javascript

vividviolet · 2017-11-27T21:32:05Z

Revising the guide to add more information on Karma, Jest and generating test coverage.

I think the recommendation for Teaspoon is out of date since there are a bunch of drawbacks compared to Karma and Jest. I've successfully worked with @macdonaldr93 and a bunch of other FEDS to move existing projects off of Teaspoon and onto Karma which enabled easier testing and code coverage to be generated.

nwtn · 2017-11-27T21:39:39Z

testing/README.md

+- For projects using [Jest](http://facebook.github.io/jest/), you do not need an additional test runner. Just use Jest to run your tests and generate test coverage. See [Coverage](#coverage) for more information on test coverage with Jest.  

- Rails projects should use [Teaspoon](https://github.com/modeset/teaspoon), which provides an excellent GUI for running tests in the browser. If you are also using [sprockets-commoner](https://github.com/Shopify/sprockets-commoner) to transpile your JavaScript, make sure to include [teaspoon-bundle](https://github.com/Shopify/teaspoon-bundle) to generate your test bundle.
+- For existing projects that already have tests written in mocha/sinon/chai, we recommend [Karma](https://karma-runner.github.io/1.0/index.html). It allows running tests in the browser and integrates well with other tools/frameworks you are likely using, such as bundlers and code coverage tools. See [Coverage](#coverage) for more information on test coverage with Karma. Feel free to instead use [Testem](https://github.com/testem/testem), another great JavaScript test runner, if you would like to try something a little different.


what’s the advantage of keeping testem here? are we using it anywhere? i'd be happy to just remove that bit.

Me too, never really made too much sense to me that we recommended two things.

I'm happy to get rid of it

nwtn · 2017-11-27T21:41:03Z

testing/README.md

+- **Jest**: You can use the built in coverage reporter by adding the `--coverage` argument when running tests. See (documentation)[https://facebook.github.io/jest/docs/en/cli.html#coverage].
+- **Other**: We recommend using [nyc](https://github.com/istanbuljs/nyc) as a standalone command line tool to generate coverage.
+
+It is best practice to generate coverage for all relevant JS files as opposed to only tested files. This will files are not inadvertently missed. There are a couple of ways to generate coverage for all files:


Word missing in second sentence

nwtn · 2017-11-27T21:44:11Z

testing/README.md

+
+## Coverage
+
+Generating a test coverage report can help you identify areas of the code that are lacking tests. Coverage reports can be integrated with CodeCov and Circle CI to ensure sufficient coverage with every checkin. See the [CodeCov Integration Guide](https://vault.shopify.com/Codecov-integration) for more information.


might be worth mentioning that we’re not striving for 100% coverage usually. as you say, generating coverage reports should help us find untested code, but usually we don’t set a hard number like “we must have 90% coverage”.

Some teams do have a strict coverage threshold, I think it's fair for that to be decided on a team-by-team basis.

How about:

Generating a test coverage report can help you identify untested code. Coverage reports can be integrated with CodeCov and Circle CI to ensure team-specific coverage targets are met with every checkin.

lemonmade · 2017-11-29T14:39:18Z

testing/README.md


+If your project is using [ReactJS](https://reactjs.org/) or is built with [Sewing-kit](https://github.com/Shopify/sewing-kit), we recommend using the following tools:
+- [Jest](http://facebook.github.io/jest/) for the test framework. 
+- [Enzyme](http://airbnb.io/enzyme/) for handling spies and mocks.


Enzyme doesn't handle spies or mocks, it is purely for giving you a nice API for dealing with React components rendered in test. Jest provides its own spying/ mocking utilities that we use.

Oops, that's a mistake. I'll clarify.

lemonmade · 2017-11-29T14:40:15Z

testing/README.md

- For pure-JavaScript projects that do not need access to the DOM, avoid using a test runner and simply run the CLI for your test framework instead (all of the projects listed above have some form of CLI). If you need a simple DOM without actual rendering, you can also use [JSDom](https://github.com/tmpvar/jsdom) to construct a fake DOM.
-
- For Non-Rails projects that require a full DOM, we recommend [Karma](https://karma-runner.github.io/1.0/index.html). It integrates well with other tools you are likely using, such as bundlers and code coverage tools. Feel free to instead use [Testem](https://github.com/testem/testem), another great JavaScript test runner, if you would like to try something a little different.
+- For projects using [Jest](http://facebook.github.io/jest/), you do not need an additional test runner. Just use Jest to run your tests and generate test coverage. See [Coverage](#coverage) for more information on test coverage with Jest.  


Good to call out here: Jest will default to providing you a "fake" browser environment with JSDom. This is enough in most cases, but if you want to test things in a true browser environment it will not be sufficient to rely on Jest alone (unsure if Jest can be run in the browser yet, it couldn't in the past).

lemonmade · 2017-11-29T14:40:44Z

testing/README.md

+- For projects using [Jest](http://facebook.github.io/jest/), you do not need an additional test runner. Just use Jest to run your tests and generate test coverage. See [Coverage](#coverage) for more information on test coverage with Jest.  

- Rails projects should use [Teaspoon](https://github.com/modeset/teaspoon), which provides an excellent GUI for running tests in the browser. If you are also using [sprockets-commoner](https://github.com/Shopify/sprockets-commoner) to transpile your JavaScript, make sure to include [teaspoon-bundle](https://github.com/Shopify/teaspoon-bundle) to generate your test bundle.
+- For existing projects that already have tests written in mocha/sinon/chai, we recommend [Karma](https://karma-runner.github.io/1.0/index.html). It allows running tests in the browser and integrates well with other tools/frameworks you are likely using, such as bundlers and code coverage tools. See [Coverage](#coverage) for more information on test coverage with Karma. Feel free to instead use [Testem](https://github.com/testem/testem), another great JavaScript test runner, if you would like to try something a little different.


Me too, never really made too much sense to me that we recommended two things.

lemonmade · 2017-11-29T14:41:30Z

testing/README.md

+
+## Coverage
+
+Generating a test coverage report can help you identify areas of the code that are lacking tests. Coverage reports can be integrated with CodeCov and Circle CI to ensure sufficient coverage with every checkin. See the [CodeCov Integration Guide](https://vault.shopify.com/Codecov-integration) for more information.


Some teams do have a strict coverage threshold, I think it's fair for that to be decided on a team-by-team basis.

lemonmade · 2017-11-29T14:42:12Z

testing/README.md

+  includeAllSources: true
+}
+```
+- **Jest**: If you are using [Sewing-kit](https://github.com/Shopify/sewing-kit), coverage for all files should be enabled automatically. Otherwise, you can configure the [`collectCoverageFrom`](http://facebook.github.io/jest/docs/en/configuration.html#collectcoveragefrom-array) option:


What is otherwise here? When is coverage not collected?

I meant to say, if you're not using Sewing-kit, you might need to set the collectCoverageFrom option yourself.

lemonmade · 2017-11-29T14:42:38Z

testing/README.md

+```
+- **Other**: Alternatively, you can use a single test entry file that loads all relevant JS modules. This is not recommended since the entry file would have to be updated whenever a new module is added.
+
+If you are writing code in ES6 and are bundling your code, it is best to generate coverage on non-bundled and non-transpiled code for better understandability. This should already be supported by default if you are using Webpack + Babel to bundle your code. If you are using Browserify for bundling, you might need to use the `es2016` preset along with the [browserify-istanbul plugin](https://github.com/devongovett/browserify-istanbul) to generate coverage for non-bundled and non-transpiled code. For example: 


I don't think we do(/should) use browserify on many projects.

I'm not sure how many projects are using Browserify but Assets does use it and we had to do a bit of extra work to get coverage working correctly. Just wanted to document it here in case other people run into the same roadblock.

…cific

vividviolet · 2017-12-18T22:16:25Z

@lemonmade I've addressed all issues. Can you take another look? Hoping to merge this as @macdonaldr93 is looking to create a link to this JS coverage info on the Vault page for CodeCov Integration

Update JS testing guide to include Jest and info on generating coverage

e2aff82

vividviolet requested review from lemonmade, macdonaldr93 and nwtn November 27, 2017 21:32

nwtn reviewed Nov 27, 2017

View reviewed changes

lemonmade requested changes Nov 29, 2017

View reviewed changes

Make changes according to PR review

144fd37

vividviolet requested a review from TheEnzoMar December 14, 2017 19:08

Update Coverage guide to include that coverage target can be team-spe…

8ad5c47

…cific

lemonmade approved these changes Dec 20, 2017

View reviewed changes

vividviolet changed the title ~~[WIP] Update JS testing guide to include Jest and info on generating coverage~~ Update JS testing guide to include Jest and info on generating coverage Dec 21, 2017

vividviolet merged commit 4572a3c into master Dec 21, 2017


		## Coverage

		Generating a test coverage report can help you identify areas of the code that are lacking tests. Coverage reports can be integrated with CodeCov and Circle CI to ensure sufficient coverage with every checkin. See the [CodeCov Integration Guide](https://vault.shopify.com/Codecov-integration) for more information.

Conversation

vividviolet commented Nov 27, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vividviolet commented Dec 18, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants