Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Regression tests on Node #410

Closed
wants to merge 1 commit into from

3 participants

@twada

We need to test QUnit HEAD in lib directory every time to detect regressions immediately.

First I tried to use node-qunit for this purpose, however node-qunit uses git-submoduled QUnit, so the QUnit object run by node-qunit is not fresh enough. It is not good for development of QUnit itself. Using qunit npm module does not mean testing qunitjs. So I wrote a simple grunt task to test QUnit HEAD on Node.

Please note: require('qunitjs') fails until merging #401

When merging is done, running against test/test.js and test/deepEqual.js then result is:

>> 308 assertions in (1709ms), passed: 296, failed: 12

12 failures are due to absence of browser related objects(window, document).

It would be better to use JUnitLogger reporter or TAP reporter to integarate with Jenkins.

This pull-request is related to #401 and #400

@JamesMGreene
Collaborator

Why not just run this grunt task as part of our local development, too, though? Seems like it would be valid to run this just as often as running the PhantomJS browser-based tests in Grunt, IMHO.

@JamesMGreene
Collaborator

It would also be nice to have a configuration option here to specify the test list, rather than hard-coding them in at the end with require calls.

@jzaefferer
Owner

@JamesMGreene would be great if you could put together a new PR that merges this one, #401, runs this new task as part of the default task and fixes the 12 failures.

Regarding the failures: In Rhino etc. there was no setTimeout, so checking for that inside our tests was enough to get them to pass. Node has setTimeout, but no window and document globals. Need to figure out if we just need additional checks for those or if we should replace or enhance the setTimeout checks.

@JamesMGreene
Collaborator

@twada: You need to sign the CLA before we can work on merging this PR. Please add a comment here when you've done so. Thanks!

@twada

@JamesMGreene Thanks! I signed. :ok_hand:

@JamesMGreene
Collaborator

@twada: I'm not sure how this worked for you before as I am getting issues immediately upon hooking this up:

Running "test-on-node" task
[DEBUGGING] QUnit:
{ module: [Function],
  asyncTest: [Function],
  test: [Function],
  expect: [Function],
  start: [Function],
  stop: [Function],
  ok: [Function],
  equal: [Function],
  notEqual: [Function],
  propEqual: [Function],
  notPropEqual: [Function],
  deepEqual: [Function],
  notDeepEqual: [Function],
  strictEqual: [Function],
  notStrictEqual: [Function],
  throws: [Function],
  raises: [Function],
  equals: [Function],
  same: [Function],
  begin: [Function],
  done: [Function],
  log: [Function],
  testStart: [Function],
  testDone: [Function],
  moduleStart: [Function],
  moduleDone: [Function]
}
Warning: Cannot set property 'autorun' of undefined Use --force to continue.

Aborted due to warnings.

This is due to the fact that we are only exporting QUnit.constructor.prototype but not QUnit itself as in the browser exports — which provides the other important methods we need for Node support, e.g.: config, load, etc.

@jzaefferer
Owner

@JamesMGreene that export looks like a bug introcued here: fb9ddcd#L3L2182 - based on the commit comment and the lack of node-based testing, we should just revert that. Can you take care of that?

@JamesMGreene
Collaborator

@jzaefferer: I tried that (seemed like the obvious thing to do) before I posted the comment but it seems like it is still missing some things (unless I'm missing something :smile:), e.g. test, asyncTest, module, expect, the lifecycle logging hook functions, etc.:

Running "test-on-node" task
[DEBUGGING] QUnit:
{ constructor: [Function: F],
  urlParams: {},
  isLocal: true,
  assert:
   { ok: [Function],
     equal: [Function],
     notEqual: [Function],
     propEqual: [Function],
     notPropEqual: [Function],
     deepEqual: [Function],
     notDeepEqual: [Function],
     strictEqual: [Function],
     notStrictEqual: [Function],
     throws: [Function] },
  config:
   { queue: [],
     blocking: true,
     hidepassed: false,
     reorder: true,
     altertitle: true,
     requireExpects: false,
     urlConfig: [ [Object], [Object] ],
     modules: {},
     begin: [],
     done: [],
     log: [],
     testStart: [],
     testDone: [],
     moduleStart: [],
     moduleDone: [],
     filter: undefined,
     module: undefined,
     testNumber: null,
     autorun: true },
  init: [Function],
  reset: [Function],
  triggerEvent: [Function],
  is: [Function],
  objectType: [Function],
  push: [Function],
  pushFailure: [Function],
  url: [Function],
  extend: [Function: extend],
  id: [Function: id],
  addEvent: [Function: addEvent],
  addClass: [Function: addClass],
  hasClass: [Function: hasClass],
  removeClass: [Function: removeClass],
  load: [Function],
  equiv: [Function],
  jsDump:
   { parse: [Function],
     typeOf: [Function],
     separator: [Function],
     indent: [Function],
     up: [Function],
     down: [Function],
     setParser: [Function],
     quote: [Function: quote],
     literal: [Function: literal],
     join: [Function: join],
     depth: 1,
     parsers:
      { window: '[Window]',
        document: '[Document]',
        error: [Function],
        unknown: '[Unknown]',
        null: 'null',
        undefined: 'undefined',
        function: [Function],
        array: [Function: array],
        nodelist: [Function: array],
        arguments: [Function: array],
        object: [Function],
        node: [Function],
        functionArgs: [Function],
        key: [Function: quote],
        functionCode: '[code]',
        attribute: [Function: quote],
        string: [Function: quote],
        date: [Function: quote],
        regexp: [Function: literal],
        number: [Function: literal],
        boolean: [Function: literal] },
     HTML: false,
     indentChar: '  ',
     multiline: true },
  diff: [Function]
}
Warning: Object #<F> has no method 'begin' Use --force to continue.

Aborted due to warnings.
@JamesMGreene
Collaborator

P.S. Figured this out... it's because our extend doesn't go into the prototype.

@JamesMGreene JamesMGreene referenced this pull request from a commit
@JamesMGreene JamesMGreene Enabling Node.js support.
Closes #401.
Closes #410.
8db51fc
@JamesMGreene JamesMGreene referenced this pull request from a commit
@twada twada Adding a new Grunt task to enable Node.js testing.
Closes #401.
Closes #410.
e4a6db3
@JamesMGreene
Collaborator

Replaced by PR #458.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Feb 3, 2013
  1. @twada
This page is out of date. Refresh to see the latest.
Showing with 70 additions and 24 deletions.
  1. +46 −0 grunt.js
  2. +1 −1  test/deepEqual.js
  3. +23 −23 test/test.js
View
46 grunt.js
@@ -96,6 +96,52 @@ grunt.registerTask( "testswarm", function( commit, configFile ) {
);
});
+
+grunt.registerTask( "test-on-node", function() {
+ var done = this.async(),
+ QUnit = require("./qunit/qunit");
+ QUnit.begin(function() {
+ grunt.log.ok('BEGIN');
+ });
+ QUnit.log(function( details ) {
+ if ( details.result ) {
+ return;
+ }
+ var message = "name: " + details.name + " module: " + details.module + " message: " + details.message;
+ grunt.log.error(message);
+ });
+ QUnit.done(function( details ) {
+ var succeeded = (details.failed === 0),
+ message = details.total + " assertions in (" + details.runtime + "ms), passed: " + details.passed + ", failed: " + details.failed;
+ if ( succeeded ) {
+ grunt.log.ok(message);
+ } else {
+ grunt.log.error(message);
+ }
+ done( succeeded );
+ });
+ QUnit.config.autorun = false;
+ QUnit.load();
+
+ var extend = function ( a, b ) {
+ for ( var prop in b ) {
+ if ( b[ prop ] === undefined ) {
+ delete a[ prop ];
+ } else if ( prop !== "constructor" ) {
+ a[ prop ] = b[ prop ];
+ }
+ }
+ return a;
+ };
+
+ extend(global, QUnit);
+ global.QUnit = QUnit;
+
+ require("./test/test");
+ require("./test/deepEqual");
+});
+
+
grunt.registerTask('default', 'lint qunit');
};
View
2  test/deepEqual.js
@@ -1,4 +1,4 @@
-module("equiv");
+QUnit.module("equiv");
test("Primitive types and constants", function () {
View
46 test/test.js
@@ -81,7 +81,7 @@ QUnit.test( "QUnit.assert compatibility", 5, function( assert ) {
assert.notStrictEqual( window.assert, QUnit.assert, "Assert does not get exposed as a global variable" );
});
-module("setup test", {
+QUnit.module("setup test", {
setup: function() {
ok(true);
}
@@ -96,7 +96,7 @@ test("module with setup, expect in test call", 2, function() {
ok(true);
});
-module("<script id='qunit-unescaped-module'>'module';</script>", {
+QUnit.module("<script id='qunit-unescaped-module'>'module';</script>", {
setup: function() {
},
teardown: function() {
@@ -123,7 +123,7 @@ test("<script id='qunit-unescaped-test'>'test';</script>", 1, function() {
var state;
-module("setup/teardown test", {
+QUnit.module("setup/teardown test", {
setup: function() {
state = true;
ok(true);
@@ -161,7 +161,7 @@ test("module with setup/teardown", function() {
ok(true);
});
-module("setup/teardown test 2");
+QUnit.module("setup/teardown test 2");
test("module without setup/teardown", function() {
expect(1);
@@ -170,7 +170,7 @@ test("module without setup/teardown", function() {
var orgDate;
-module("Date test", {
+QUnit.module("Date test", {
setup: function() {
orgDate = Date;
window.Date = function () {
@@ -191,7 +191,7 @@ test("sample test for Date test", function () {
if (typeof setTimeout !== 'undefined') {
state = 'fail';
-module("teardown and stop", {
+QUnit.module("teardown and stop", {
teardown: function() {
equal(state, "done", "Test teardown.");
}
@@ -231,7 +231,7 @@ test("parameter passed to start decrements semaphore n times", function() {
}, 18);
});
-module("async setup test", {
+QUnit.module("async setup test", {
setup: function() {
stop();
setTimeout(function() {
@@ -247,7 +247,7 @@ asyncTest("module with async setup", function() {
start();
});
-module("async teardown test", {
+QUnit.module("async teardown test", {
teardown: function() {
stop();
setTimeout(function() {
@@ -263,7 +263,7 @@ asyncTest("module with async teardown", function() {
start();
});
-module("asyncTest");
+QUnit.module("asyncTest");
asyncTest("asyncTest", function() {
expect(2);
@@ -311,7 +311,7 @@ test("test synchronous calls to stop", 2, function() {
});
}
-module("save scope", {
+QUnit.module("save scope", {
setup: function() {
this.foo = "bar";
},
@@ -324,7 +324,7 @@ test("scope check", function() {
deepEqual(this.foo, "bar");
});
-module("simple testEnvironment setup", {
+QUnit.module("simple testEnvironment setup", {
foo: "bar",
// example of meta-data
bugid: "#5311"
@@ -340,7 +340,7 @@ test("testEnvironment reset for next test",function() {
deepEqual(this.foo, "bar");
});
-module("testEnvironment with object", {
+QUnit.module("testEnvironment with object", {
options:{
recipe:"soup",
ingredients:["hamster","onions"]
@@ -359,7 +359,7 @@ test("testEnvironment reset for next test",function() {
});
-module("testEnvironment tests");
+QUnit.module("testEnvironment tests");
function makeurl() {
var testEnv = QUnit.current_testEnvironment;
@@ -373,7 +373,7 @@ test("makeurl working",function() {
equal( makeurl(), 'http://example.com/search?q=a%20search%20test', 'makeurl returns a default url if nothing specified in the testEnvironment');
});
-module("testEnvironment with makeurl settings", {
+QUnit.module("testEnvironment with makeurl settings", {
url: 'http://google.com/',
q: 'another_search_test'
});
@@ -381,7 +381,7 @@ test("makeurl working with settings from testEnvironment", function() {
equal( makeurl(), 'http://google.com/?q=another_search_test', 'rather than passing arguments, we use test metadata to from the url');
});
-module("jsDump");
+QUnit.module("jsDump");
test("jsDump output", function() {
equal( QUnit.jsDump.parse([1, 2]), "[\n 1,\n 2\n]" );
equal( QUnit.jsDump.parse({top: 5, left: 0}), "{\n \"left\": 0,\n \"top\": 5\n}" );
@@ -391,7 +391,7 @@ test("jsDump output", function() {
}
});
-module("assertions");
+QUnit.module("assertions");
test("propEqual", 5, function( assert ) {
var objectCreate = Object.create || function ( origin ) {
@@ -560,7 +560,7 @@ test("raises", 9, function() {
if (typeof document !== "undefined") {
-module("fixture");
+QUnit.module("fixture");
test("setup", function() {
expect(0);
document.getElementById("qunit-fixture").innerHTML = "foobar";
@@ -586,7 +586,7 @@ test("running test name displayed", function() {
setTimeout( function() { start(); }, n );
};
- module("timing", {
+ QUnit.module("timing", {
setup: function() {
if ( delayNextSetup ) {
delayNextSetup = false;
@@ -616,7 +616,7 @@ test("running test name displayed", function() {
}
-module("custom assertions");
+QUnit.module("custom assertions");
(function() {
function mod2(value, expected, message) {
var actual = value % 2;
@@ -629,7 +629,7 @@ module("custom assertions");
})();
-module("recursions");
+QUnit.module("recursions");
function Wrap(x) {
this.wrap = x;
@@ -741,7 +741,7 @@ test('Circular reference - test reported by soniciq in #105', function() {
(function() {
var reset = QUnit.reset;
- module("reset");
+ QUnit.module("reset");
test("reset runs assertions", function() {
expect(0);
QUnit.reset = function() {
@@ -762,7 +762,7 @@ function testAfterDone() {
QUnit.config.done = [];
// Because when this does happen, the assertion count parameter doesn't actually
// work we use this test to check the assertion count.
- module("check previous test's assertion counts");
+ QUnit.module("check previous test's assertion counts");
test('count previous two test\'s assertions', function () {
var tests = getPreviousTests(/^ensure has correct number of assertions/, /^Synchronous test after load of page$/);
@@ -773,7 +773,7 @@ function testAfterDone() {
QUnit.config.done = [];
QUnit.done(secondAfterDoneTest);
- module("Synchronous test after load of page");
+ QUnit.module("Synchronous test after load of page");
asyncTest('Async test', function() {
start();
Something went wrong with that request. Please try again.