Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing: Update documentation to reflect migration to Jest #16943

Merged
merged 2 commits into from Sep 28, 2017

Conversation

@gziolo
Copy link
Member

commented Aug 5, 2017

2 months back I suggested that we migrate our tests to run with Jest instead of Mocha. I published comparison of those two test tools and got very positive feedback about this whole idea. Public version of this post can be found here. With #16119 and #14364 we want to make it happen. Coding part is up and running and needs only final polishing. However we still need to update our documentation to reflect all changes introduced. At the same time we can revisit what we have at the moment and make it a way much better. In this PR I'm proposing a few new paragraphs which in my opinion will explain better how we test code in Calypso. I would like to make this PR team effort to increase awareness about importance of all types of tests. Thanks for all contributions in advance.

There is only one thing missing that I plan to do to make transition smoother. I want to write a blog post explaining what has really changed and how migration were performed.

@matticbot

This comment has been minimized.

Copy link

commented Aug 5, 2017

@matticbot matticbot added [Size] XL and removed [Size] L labels Aug 7, 2017

> npm run test-server
> # run a configuration customized to work with continuous integration
> npm run test-server:ci
> # run tests in watch mode, by default it executes tests only for the modified files

This comment has been minimized.

Copy link
@gziolo

gziolo Aug 7, 2017

Author Member

Word 'only' should be at the end here and in the next section.


### Client side tests

This test configuration unit and component tests that verify code located in `client` top level folder.

This comment has been minimized.

Copy link
@gziolo

gziolo Aug 7, 2017

Author Member

Word 'contains' is missing in the first sentence.


They are executed on every push on continuous integration (CircleCI). This is why all individual tests need to be blazing fast. Please note that network connection is disabled for this configuration.

_Check also how to write [unit components](unit-tests.md) and [component tests](component-tests.md)._

This comment has been minimized.

Copy link
@gziolo

gziolo Aug 7, 2017

Author Member

Should be 'unit tests', not 'unit components'.


They play role of system tests, because they are executed against the entire integrated WordPress.com system.

All those tests can't be found in this repository. They live in [Automattic/wp-e2e-tests](https://github.com/Automattic/wp-e2e-tests) repository. See [Introducing our WordPress.com Automated e2e Tests](https://developer.wordpress.com/2016/05/12/automated-e2e-tests/) post to find out more.

This comment has been minimized.

Copy link
@gziolo

gziolo Aug 7, 2017

Author Member

Two first sentences should be combined.


__What tools and libraries are used?__

We use [Jest](https://facebook.github.io/jest/) testing tool to execute all test configurations located in Calypso repository. We recommend using its very flexible [API](https://facebook.github.io/jest/docs/en/api.html) with [expect matchers](https://facebook.github.io/jest/docs/en/expect.html) and [mock functions](https://facebook.github.io/jest/docs/en/mock-function-api.html).

This comment has been minimized.

Copy link
@gziolo

gziolo Aug 7, 2017

Author Member

We recommend using > It's highly recommend to use?


Calypso has a lot of React UI components. (Try for example running `find -name *.jsx` under the `client` folder). This combined with the fact that UI tests are among the more difficult things to (unit) test, a guide or checklist for how to make it as easy and focused as possible seems appropriate.

## [Getting started](#getting-started)

To run all current tests, run `npm test` in the root source folder. You can also run individual tests. Check [How to run single test runner](https://github.com/Automattic/wp-calypso/blob/master/test/README.md#how-to-run-single-test-runner) documentation for more details.
To run all current client side tests, run `npm run test-client` in the root source folder. You can also run individual tests. Check [How to run single test runner](https://github.com/Automattic/wp-calypso/blob/master/docs/testing.md) documentation for more details.

This comment has been minimized.

Copy link
@gziolo

gziolo Aug 7, 2017

Author Member

Link needs to be updated.

@@ -0,0 +1,3 @@
# Unit Tests

TBD

This comment has been minimized.

Copy link
@gziolo

gziolo Aug 7, 2017

Author Member

@aduth or @ockham do you have any resources from GM class you taught last year that we could use here? 😃

This comment has been minimized.

Copy link
@aduth

aduth Aug 11, 2017

Member

Are you wanting a blurb about what and why we unit test?

See also: p4TIVU-532-p2

Don't really have a good piece of longer-form text already prepared, but I might be able to put one together if you'd like.

This comment has been minimized.

Copy link
@gziolo

gziolo Aug 12, 2017

Author Member

Are you wanting a blurb about what and why we unit test?

It would be great to put something short for start. We can team up when I'm back from AFK and come up with a comprehensive document based on the slides you shared in the link (BTW, they are great and in my opinion they should be shared with the public, too). I've found recently also another excellent guide which we can use as a reference: http://acco.io/a-practical-guide-to-testing-react-apps/.


_Check also how to write [unit components](unit-tests.md) and [component tests](component-tests.md)._

### Integration tests

This comment has been minimized.

Copy link
@gziolo

gziolo Aug 7, 2017

Author Member

@gwwar feel free to add/update this section as you should have better idea how it should really be used.


They run daily on continuous integration (CircleCI), because they can use network connection or memory intensive processing and therefore can have longer runtime.

### End-to-end tests

This comment has been minimized.

Copy link
@gziolo

gziolo Aug 7, 2017

Author Member

@lancewillett, @alisterscott and @hoverduck It would be great to provide more details here to encourage people to start writing e2e tests, too. It would be awesome to make it as a requirement when exposing new festures to the users.

@markryall

This comment has been minimized.

Copy link
Contributor

commented Aug 9, 2017

This is looking great @gziolo!

An excellent introduction to the motivation and nature of the various flavors of testing.

It will be great to continue adding more detail around some of the chosen conventions for unit testing specific kinds of things such as components, selectors, middleware with links to some examples. These conventions will evolve as we may make more use of jest features such as snapshots.

@hoverduck
Copy link
Member

left a comment

Looks great, thanks for putting this together!


### Integration tests

This test configuration contains integration tests that verify code located in `bin`, `client`, `server` and `test` top level folders. They should test how a group of components or a larger part of business logic works together.

This comment has been minimized.

Copy link
@gwwar

gwwar Aug 11, 2017

Member

I think we might want to note that this is meant for services/flows that need to be up that we maintain.

Anything visual is better done as a full e2e test in https://github.com/Automattic/wp-e2e-tests

This comment has been minimized.

Copy link
@gziolo

gziolo Aug 12, 2017

Author Member

Thanks, I will include this note, too.

@gwwar
gwwar approved these changes Aug 11, 2017
Copy link
Member

left a comment

thanks @gziolo these look great

@dmsnell
Copy link
Contributor

left a comment

These are great improvements. I don't think we should wait on unfinished work (like about unit testing) but should merge soon and continue to iterate on the docs. I'd be happy to try and squeeze out some time to talk about unit tests too.

@nb

This comment has been minimized.

Copy link
Member

commented Aug 18, 2017

Agreed, let's merge and see where it takes us 👍

@gziolo

This comment has been minimized.

Copy link
Member Author

commented Aug 22, 2017

I should finish migration summary quite soon. I will provide examples how to write tests taking into account Jest API, mockery deprecation and jsdom built-in support to simulate browser. Once it's ready we should be good to go with all 3 PRs.

@markryall markryall removed their request for review Aug 28, 2017

@gziolo

This comment has been minimized.

Copy link
Member Author

commented Sep 20, 2017

We decided to put this one on hold. I will open another PR with the parts that fit to the existing setup with Mocha.

@gziolo gziolo closed this Sep 20, 2017

@gziolo gziolo deleted the update/testing-docs branch Sep 20, 2017

@gziolo

This comment has been minimized.

Copy link
Member Author

commented Sep 25, 2017

Reopening after Facebook announced relicensing under the MIT license: https://code.facebook.com/posts/300798627056246/relicensing-react-jest-flow-and-immutable-js/.

@gziolo gziolo restored the update/testing-docs branch Sep 25, 2017

@gziolo gziolo reopened this Sep 25, 2017

@gziolo gziolo force-pushed the update/testing-docs branch from b513bca to c68a736 Sep 25, 2017

@gziolo

This comment has been minimized.

Copy link
Member Author

commented Sep 25, 2017

Rebased with no changes. It should be still good to go.

@gziolo gziolo merged commit cfc4b90 into master Sep 28, 2017

1 check passed

ci/circleci Your tests passed on CircleCI!
Details

@gziolo gziolo deleted the update/testing-docs branch Sep 28, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.