Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Travis CI integration #1201

Closed
wants to merge 1 commit into from

5 participants

@mark-rushakoff

This commit includes everything necessary to run the test suite in PhantomJs on Travis CI.

You can see a green build at http://travis-ci.org/#!/mark-rushakoff/backbone

@braddunbar
Collaborator

I'm not sure where @jashkenas stands on this, but if we're going to run tests from the console I'd rather it be rolled up in something like grunt so that we aren't maintaining it. Will grunt run on travis? If so, can we just include it in package.json and point travis to a script that does grunt qunit?

@mark-rushakoff

I don't know enough about node/npm to say so definitively, but it does look like you would be able to just drop grunt in (via package.json) and run qunit.

I didn't know of grunt until just now, and glancing over its code, I feel like he's offering a more full-featured suite of tools at the cost of more configuration. On the other hand, your concern about maintenance is very valid, and I haven't yet solved that problem in my project (OpenPhantomScripts, from which the code in this pull request came).

I won't be offended if you choose grunt over my project -- I'd just really like to see that Travis "build: passing" badge on the front page of Underscore and Backbone.

In the meantime, I'm going to do some research into how to alleviate the maintenance concern you have with my code (if anyone has any comments, please chime in on the issue on OpenPhantomScripts).

@jashkenas
Owner

I'm a little confused -- why would we want to test Backbone outside of a browser? ... given that it's intended to run in the browser, and we already test commits and patches in browsers. What do we gain?

@tbranyen
Collaborator

@jashkenas headless testing with grunt would be nice since its something you get "for free", it takes no added work other than pointing to the qunit folder and then you could put that in front of the min process to stop it from building if there are any errors.

i don't see it as anything other than convenience.

@mark-rushakoff

PhantomJs is a headless webkit browser. It should behave (roughly) the same as Safari or Chrome. So running the tests in Phantom isn't "outside a browser", it's "in another browser."

While I do believe that you do manually test commits, by not having a published CI, I feel like you're asking the consumers of Underscore and Backbone to take it on faith that the last commit has been tested. Or you're asking us to run the tests ourselves before we put the library in our code.

When I see the green "build passed" badge on a Github project, I am reasonably assured that the code from the latest commit works as intended.

Maybe a compromise is to put a direct link to the tests in the README?

@jashkenas
Owner

Yep -- there are direct links to the live test pages at the top of the homepage (which serves as the real README).

When I see the green "build passed" badge on a Github project, I am
reasonably assured that the code from the latest commit works as intended.

... I think you're putting overmuch faith in the existence of tests then. Our releases always pass the test suite in major browsers, but we nonetheless frequently have bugs in untested edge cases.

@jashkenas jashkenas closed this
@jashkenas jashkenas reopened this
@jashkenas
Owner

Reopening ...

Now that GitHub pull requests support Travis badges, there's a pretty solid benefit to potentially adding Travis support.

https://github.com/blog/1227-commit-status-api

@philfreo

+1 for using grunt qunit. grunt could also replace the Rakefile so the project is fully JS-based.

http://gruntjs.com/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Apr 7, 2012
  1. @mark-rushakoff

    Travis CI integration

    mark-rushakoff authored
This page is out of date. Refresh to see the latest.
Showing with 113 additions and 0 deletions.
  1. +98 −0 .phantom-qunit.js
  2. +15 −0 .travis.yml
View
98 .phantom-qunit.js
@@ -0,0 +1,98 @@
+// Part of OpenPhantomScripts
+// http://github.com/mark-rushakoff/OpenPhantomScripts
+
+// Copyright (c) 2012 Mark Rushakoff
+
+// Permission is hereby granted, free of charge, to any person obtaining a copy
+// of this software and associated documentation files (the "Software"), to
+// deal in the Software without restriction, including without limitation the
+// rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+// sell copies of the Software, and to permit persons to whom the Software is
+// furnished to do so, subject to the following conditions:
+
+// The above copyright notice and this permission notice shall be included in
+// all copies or substantial portions of the Software.
+
+// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+// FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+// IN THE SOFTWARE.
+
+var fs = require("fs");
+var args, url, lengthOkay, appName, system;
+try {
+ system = require("system");
+ // if we got here, we are on PhantomJS 1.5+
+ args = system.args;
+ lengthOkay = (args.length === 2);
+ appName = args[0];
+ url = args[1];
+} catch (e) {
+ // otherwise, assume PhantomJS 1.4
+ args = phantom.args;
+ lengthOkay = (args.length === 1);
+ appName = 'phantom-qunit.js'
+ url = args[0];
+}
+
+if (!lengthOkay) {
+ printError("Usage: " + appName + " URL");
+ phantom.exit(1);
+}
+
+function printError(message) {
+ fs.write("/dev/stderr", message + "\n", "w");
+}
+
+var page = require("webpage").create();
+
+var attachedDoneCallback = false;
+page.onResourceReceived = function() {
+ // Without this guard, I was occasionally seeing the done handler
+ // pushed onto the array multiple times -- it looks like the
+ // function was queued up several times, depending on the server.
+ if (!attachedDoneCallback) {
+ attachedDoneCallback = page.evaluate(function() {
+ if (window.QUnit) {
+ window.QUnit.config.done.push(function(obj) {
+ console.log("Tests passed: " + obj.passed);
+ console.log("Tests failed: " + obj.failed);
+ console.log("Total tests: " + obj.total);
+ console.log("Runtime (ms): " + obj.runtime);
+ window.phantomComplete = true;
+ window.phantomResults = obj;
+ });
+
+ return true;
+ }
+
+ return false;
+ });
+ }
+}
+
+page.onConsoleMessage = function(message) {
+ console.log(message);
+}
+
+page.open(url, function(success) {
+ if (success === "success") {
+ if (!attachedDoneCallback) {
+ printError("Phantom callbacks not attached in time. See http://github.com/mark-rushakoff/OpenPhantomScripts/issues/1");
+ phantom.exit(1);
+ }
+
+ setInterval(function() {
+ if (page.evaluate(function() {return window.phantomComplete;})) {
+ var failures = page.evaluate(function() {return window.phantomResults.failed;});
+ phantom.exit(failures);
+ }
+ }, 250);
+ } else {
+ printError("Failure opening " + url);
+ phantom.exit(1);
+ }
+});
View
15 .travis.yml
@@ -0,0 +1,15 @@
+# We aren't actually using any particular language here, so let's use Ruby
+# since that's probably the most likely Travis worker to be available
+language: ruby
+rvm:
+ - 1.8.7
+notifications:
+ email: false
+before_script:
+ # Travis needs this stuff to start Phantom1.4...
+ # When they get Phantom1.5 this section can go away
+ - "export DISPLAY=:99.0"
+ - "sh -e /etc/init.d/xvfb start"
+script:
+ # You can put the script in your repo as .phantom-qunit so it's hidden
+ - "phantomjs .phantom-qunit.js test/test.html"
Something went wrong with that request. Please try again.