Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Add integration testing guide #452

Closed
wants to merge 2 commits into
from

Conversation

Projects
None yet
9 participants
Contributor

joliss commented Apr 16, 2013

@kategengler and I paired and made a testing guide. More to come, but this should be a good start.

@joefiorini joefiorini referenced this pull request Apr 17, 2013

Closed

Ember.js testing guide #398

Owner

stefanpenner commented Apr 17, 2013

@trek @tomdale et al, you feedback and comments are welcome

Owner

trek commented Apr 17, 2013

Looks good to me. I want to have an "ecosystem" section somewhere in the guides for common tasks like build tools, testing, working with 3rd party libraries. Tasks not specific to Ember, but ones that many people are unclear on. We could extract @tomdale's use of moment.js in the screencast as a good example of shopping out to external libs.

Owner

trek commented Apr 17, 2013

Hm. I'd also add a single example test? Maybe something related to the starter kit?

Contributor

mharris717 commented Apr 17, 2013

This worked for me so far.

Made a sample site using the code.

Github: https://github.com/mharris717/ember_testing_static_example
Live: http://embertesting.s3-website-us-west-2.amazonaws.com/

Contributor

sly7-7 commented Apr 17, 2013

  • I wonder if stuff like App.deferReadiness(), App.reopen(), Store.reopen(), Router.reopen() Konacha.reset could/should be inside put the before() primitive. I don't know if there is some dependencies between these 'customizations'.
  • Stuff like declaring Fixtures, or using something like sinon.js to fake the server, how/where can we code this ? I think it's in the before()/ beforeEach(), but I'm not sure. Perhaps having an example of dealing with "mocking backend" would be helpfull.

@thomasboyt thomasboyt referenced this pull request Apr 18, 2013

Closed

Ecosystem Documentation #455

Contributor

gilesbowkett commented Apr 19, 2013

I forked @joliss's branch and adapted some of @mharris717's code:

https://github.com/gilesbowkett/website/blob/testing-guide/source/guides/testing/integration-testing.md

Also some writing tweaks. Not sure if I should make an additional PR or what.

Owner

stefanpenner commented Apr 19, 2013

@gilesbowkett u can submit a PR to her branch.

@gilesbowkett gilesbowkett referenced this pull request in joliss/website Apr 19, 2013

Closed

Expanded explanations and examples #1

Contributor

gilesbowkett commented Apr 19, 2013

@stefanpenner cheers, done

Jo Liss and Katie Gengler and others added some commits Apr 16, 2013

Contributor

joliss commented Apr 23, 2013

Hey everyone, @ebryn and I had a chat on Friday, and here's the summary:

This PR, in particular in the section "The Run Loop", suggest a synchronous testing style.

Erik wants to go straight to recommending async testing, and he's writing code to support it in the ember-testing package. "Async testing" meaning, your tests go like this: "Click stuff", "wait for absolutely everything [Ajax, timers, ...] to complete", ".then check stuff in the DOM".

The synchronous style is only usable for frontend-only tests (talking to a fixture store). This async approach would be more universal, in that it also covers JavaScript-driven full-stack integration tests (talking to a backend).

Looking at Erik's code, the implementation for async testing support (ember-testing) seems to be coming along nicely, so if this turns out solid, it's probably better to recommend the async style. We agree that async fundamentally is the way forward.

I basically have two concerns (those are not objections):

  • The "wait for absolutely everything" implementation eventually needs to be more solid and not have ugly edge cases. Right now we do stuff like hooking into $.ajax.
  • I've used Capybara+Selenium/WebKit a lot. With those tools, it's almost impossible to write a medium-sized test suite without accidentally having race conditions in your test code. Having a race condition every hundred tests or so hugely reduced the value of a test suite. I'm worried that we might run into similar kinds of problems with our async approach. If we're going to recommend an async testing style for everything, we need to be sure that it is absolutely solid, both architecturally and in practice.

Erik thinks both of these are solvable.

P.S. One tricky part when talking to a backend is setting up fixtures from the JavaScript test code, but perhaps we'll solve some of this in the coming months.

Contributor

joliss commented Apr 23, 2013

I'm out of personal bandwidth at the moment, so I'm deferring to Erik and everyone else on all the testing stuff. This PR was basically just to get our code out there (long overdue); but I want to focus on different stuff now so I won't have much resources to champion the testing ecosystem. So please nag @ebryn, not me. ;-) I trust you guys to make good decisions.

Contributor

joliss commented Apr 23, 2013

@gilesbowkett Thanks for the PR. I've cherry-picked the first commit into this PR. I think there's a lot of good stuff in the second commit, but one thing I'm worried about is recommending an ad-hoc DSL. Creating a DSL that doesn't suck is a hard problem, and I want to see (competing) stand-alone libraries emerging for that. If we put some minimal DSL in the Ember docs, it will have all kinds of limitations and end up being a maintenance headache for us. For now, I'd say, let users discover the pain points themselves and make their own DSLs.

Contributor

joliss commented Apr 23, 2013

@ all, I think there's a lot of useful stuff in this PR even if we go the async testing route. Perhaps let's get this merged and simply remove the sections we don't like? We can also always switch the syntax to QUnit later. This way we have a base that we can build on top off and add all the things people suggested, rather than have this linger forever.

If you want to throw it away and start over (or copy parts of this), that's cool too of course.

Member

kategengler commented Apr 23, 2013

@joliss @gilesbowkett I believe some DSL helpers are in the Ember testing stuff @ebryn committed.

Contributor

mharris717 commented Apr 24, 2013

On the DSL, I wrote that just to make the example site less ugly. Agreed that we shouldn't bring it in to the ember-testing package

@ebryn, I'd like to help with ember-testing. Let me know anything I can help with or anything I can contribute. I've got some available time in the near future.

ootoovak commented May 6, 2013

@ebryn I'm new to Ember (and thus the testing of it as well) but I would like to help. I am going to be using Ember-Rails and Konacha so happy to be the beginner tester from that point of view if you need that. Cheers.

Owner

tomdale commented May 10, 2013

What's the status of this? Should we merge it in? If not, can we close it until it's ready?

Contributor

joliss commented May 11, 2013

Talked to Katie; she's working on an even more awesome guide, so we can close this PR.

@joliss joliss closed this May 11, 2013

@joliss any links to share to Kaite's work?

Contributor

joliss commented May 11, 2013

It's work-in-progress from what I hear -- she'll post it soonish I assume.

@kategengler I would like to help anyway I can. Let me know if you need a newbie's point of view.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment