Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Add integration testing guide #452

Closed
wants to merge 2 commits into from

9 participants

@joliss

@kategengler and I paired and made a testing guide. More to come, but this should be a good start.

@joefiorini joefiorini referenced this pull request
Closed

Ember.js testing guide #398

@stefanpenner

@trek @tomdale et al, you feedback and comments are welcome

@trek
Owner

Looks good to me. I want to have an "ecosystem" section somewhere in the guides for common tasks like build tools, testing, working with 3rd party libraries. Tasks not specific to Ember, but ones that many people are unclear on. We could extract @tomdale's use of moment.js in the screencast as a good example of shopping out to external libs.

@trek
Owner

Hm. I'd also add a single example test? Maybe something related to the starter kit?

@sly7-7
  • I wonder if stuff like App.deferReadiness(), App.reopen(), Store.reopen(), Router.reopen() Konacha.reset could/should be inside put the before() primitive. I don't know if there is some dependencies between these 'customizations'.

  • Stuff like declaring Fixtures, or using something like sinon.js to fake the server, how/where can we code this ? I think it's in the before()/ beforeEach(), but I'm not sure. Perhaps having an example of dealing with "mocking backend" would be helpfull.

@thomasboyt thomasboyt referenced this pull request
Closed

Ecosystem Documentation #455

@gilesbowkett

I forked @joliss's branch and adapted some of @mharris717's code:

https://github.com/gilesbowkett/website/blob/testing-guide/source/guides/testing/integration-testing.md

Also some writing tweaks. Not sure if I should make an additional PR or what.

@stefanpenner

@gilesbowkett u can submit a PR to her branch.

@gilesbowkett gilesbowkett referenced this pull request in joliss/website
Closed

Expanded explanations and examples #1

Jo Liss and ... and others added some commits
@joliss

Hey everyone, @ebryn and I had a chat on Friday, and here's the summary:

This PR, in particular in the section "The Run Loop", suggest a synchronous testing style.

Erik wants to go straight to recommending async testing, and he's writing code to support it in the ember-testing package. "Async testing" meaning, your tests go like this: "Click stuff", "wait for absolutely everything [Ajax, timers, ...] to complete", ".then check stuff in the DOM".

The synchronous style is only usable for frontend-only tests (talking to a fixture store). This async approach would be more universal, in that it also covers JavaScript-driven full-stack integration tests (talking to a backend).

Looking at Erik's code, the implementation for async testing support (ember-testing) seems to be coming along nicely, so if this turns out solid, it's probably better to recommend the async style. We agree that async fundamentally is the way forward.

I basically have two concerns (those are not objections):

  • The "wait for absolutely everything" implementation eventually needs to be more solid and not have ugly edge cases. Right now we do stuff like hooking into $.ajax.
  • I've used Capybara+Selenium/WebKit a lot. With those tools, it's almost impossible to write a medium-sized test suite without accidentally having race conditions in your test code. Having a race condition every hundred tests or so hugely reduced the value of a test suite. I'm worried that we might run into similar kinds of problems with our async approach. If we're going to recommend an async testing style for everything, we need to be sure that it is absolutely solid, both architecturally and in practice.

Erik thinks both of these are solvable.

P.S. One tricky part when talking to a backend is setting up fixtures from the JavaScript test code, but perhaps we'll solve some of this in the coming months.

@joliss

I'm out of personal bandwidth at the moment, so I'm deferring to Erik and everyone else on all the testing stuff. This PR was basically just to get our code out there (long overdue); but I want to focus on different stuff now so I won't have much resources to champion the testing ecosystem. So please nag @ebryn, not me. ;-) I trust you guys to make good decisions.

@joliss

@gilesbowkett Thanks for the PR. I've cherry-picked the first commit into this PR. I think there's a lot of good stuff in the second commit, but one thing I'm worried about is recommending an ad-hoc DSL. Creating a DSL that doesn't suck is a hard problem, and I want to see (competing) stand-alone libraries emerging for that. If we put some minimal DSL in the Ember docs, it will have all kinds of limitations and end up being a maintenance headache for us. For now, I'd say, let users discover the pain points themselves and make their own DSLs.

@joliss

@ all, I think there's a lot of useful stuff in this PR even if we go the async testing route. Perhaps let's get this merged and simply remove the sections we don't like? We can also always switch the syntax to QUnit later. This way we have a base that we can build on top off and add all the things people suggested, rather than have this linger forever.

If you want to throw it away and start over (or copy parts of this), that's cool too of course.

@kategengler

@joliss @gilesbowkett I believe some DSL helpers are in the Ember testing stuff @ebryn committed.

@mharris717

On the DSL, I wrote that just to make the example site less ugly. Agreed that we shouldn't bring it in to the ember-testing package

@ebryn, I'd like to help with ember-testing. Let me know anything I can help with or anything I can contribute. I've got some available time in the near future.

@ootoovak

@ebryn I'm new to Ember (and thus the testing of it as well) but I would like to help. I am going to be using Ember-Rails and Konacha so happy to be the beginner tester from that point of view if you need that. Cheers.

@tomdale
Owner

What's the status of this? Should we merge it in? If not, can we close it until it's ready?

@joliss

Talked to Katie; she's working on an even more awesome guide, so we can close this PR.

@joliss joliss closed this
@ootoovak

@joliss any links to share to Kaite's work?

@joliss

It's work-in-progress from what I hear -- she'll post it soonish I assume.

@ootoovak

@kategengler I would like to help anyway I can. Let me know if you need a newbie's point of view.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Apr 23, 2013
  1. @joliss

    Add integration testing guide

    Jo Liss and Katie Gengler authored joliss committed
  2. @gilesbowkett @joliss

    punctuation, active voice, general writing

    gilesbowkett authored joliss committed
This page is out of date. Refresh to see the latest.
Showing with 115 additions and 0 deletions.
  1. +3 −0  data/guides.yml
  2. +112 −0 source/guides/testing/integration-testing.md
View
3  data/guides.yml
@@ -157,3 +157,6 @@ Understanding Ember.js:
#- title: "Record Lifecycle"
#url: "understanding-ember-data/record-lifecycle"
+Testing Ember Apps:
+ - title: "Integration Testing"
+ url: "testing/integration-testing"
View
112 source/guides/testing/integration-testing.md
@@ -0,0 +1,112 @@
+## Integration Testing
+
+Here is a suggested setup for JavaScript *client-side integration tests,*
+which test your Ember app in isolation from the backend. Our tests:
+
+* (re-)instantiate our app,
+* simulate user input (with jQuery), and
+* check that the results show up in the DOM correctly.
+
+We can also modify and check the state of the application's models
+directly, whenever it makes testing easier.
+
+By exercising the entire Ember app, we test that all its (individually simple)
+layers work together.
+
+Integration tests have a reputation for being slow and brittle. These
+client-side integration tests are not, however, because they do not
+require coordinating the backend and the frontend, so they can cover the
+bulk of our testing needs.
+
+### Setup
+
+We assume [Mocha](http://visionmedia.github.io/mocha/), but the instructions
+should be analogous for other testing frameworks, like
+[QUnit](http://qunitjs.com/).
+
+```javascript
+// Stop Ember from automatically scheduling run loops with setTimeout. It's
+// non-deterministic, which is bad for tests.
+Ember.testing = true;
+
+// Re-enable automatic run loops when testing is over, for easy debugging in
+// the console.
+after(function() { // after all tests have finished
+ Ember.testing = false;
+});
+
+
+App.reopen({
+ // Use a separate root element so we don't interfere with the test reporter.
+ rootElement: '#test-app-container'
+});
+
+// Wait to initialize until we are done setting up.
+App.deferReadiness();
+
+before(function(done) { // before any tests have started
+ // Now that the DOM is ready, add the root element.
+ $('body').append('<div id="test-app-container"></div>');
+
+ Ember.run(function() {
+ // We are done setting up. The app can now initialize.
+ App.advanceReadiness();
+
+ // This `before` handler blocks the test suite until the callback (done)
+ // is called. App.then fires when the app has finished initializing.
+ App.then(function() {
+ done();
+ });
+ });
+});
+
+
+// Reset the entire app before each test.
+beforeEach(function() {
+ Ember.run(function() {
+ App.reset();
+ });
+});
+
+
+// Do not muck with the URL, or URL state will leak between tests.
+App.Router.reopen({
+ location: 'none'
+});
+
+
+// Use the fixture adapter to pick up fixtures from App.Blog.FIXTURES, etc.
+App.Store.reopen({
+ adapter: DS.FixtureAdapter.extend({
+ // Make the adapter deterministic.
+ simulateRemoteResponse: false
+ })
+});
+```
+
+#### Konacha
+
+If you use [Konacha](https://github.com/jfirebaugh/konacha) (Mocha on Rails),
+remember to additionally stop Konacha from clearing out the `<body>` element:
+
+```javascript
+// Disable Konacha's resetting. We bring our own reset for Ember.
+Konacha.reset = function() { };
+```
+
+### The Run Loop
+
+Because we have disabled the automatic scheduling of run loops (`Ember.testing
+= true`), anything that affects the state of the Ember app has to be wrapped
+in `Ember.run`, or you will get an error message ("assertion failed: You have
+turned on testing mode, which disabled the run-loop's autorun. You will need
+to wrap any code with asynchronous side-effects in an Ember.run"). For
+example:
+
+```javascript
+Ember.run(function() { $('button.foo').click(); });
+// Now test that things have appeared in the DOM.
+```
+
+`Ember.run` ensures that any state changes will have finished by the time it
+returns.
Something went wrong with that request. Please try again.