Regression tests on Node #410

Closed
wants to merge 1 commit into
from

Conversation

Projects
None yet
3 participants
@twada

twada commented Feb 3, 2013

We need to test QUnit HEAD in lib directory every time to detect regressions immediately.

First I tried to use node-qunit for this purpose, however node-qunit uses git-submoduled QUnit, so the QUnit object run by node-qunit is not fresh enough. It is not good for development of QUnit itself. Using qunit npm module does not mean testing qunitjs. So I wrote a simple grunt task to test QUnit HEAD on Node.

Please note: require('qunitjs') fails until merging jquery#401

When merging is done, running against test/test.js and test/deepEqual.js then result is:

>> 308 assertions in (1709ms), passed: 296, failed: 12

12 failures are due to absence of browser related objects(window, document).

It would be better to use JUnitLogger reporter or TAP reporter to integarate with Jenkins.

This pull-request is related to jquery#401 and jquery#400

@JamesMGreene

This comment has been minimized.

Show comment Hide comment
@JamesMGreene

JamesMGreene Feb 15, 2013

Member

Why not just run this grunt task as part of our local development, too, though? Seems like it would be valid to run this just as often as running the PhantomJS browser-based tests in Grunt, IMHO.

Member

JamesMGreene commented Feb 15, 2013

Why not just run this grunt task as part of our local development, too, though? Seems like it would be valid to run this just as often as running the PhantomJS browser-based tests in Grunt, IMHO.

@JamesMGreene

This comment has been minimized.

Show comment Hide comment
@JamesMGreene

JamesMGreene Feb 15, 2013

Member

It would also be nice to have a configuration option here to specify the test list, rather than hard-coding them in at the end with require calls.

Member

JamesMGreene commented Feb 15, 2013

It would also be nice to have a configuration option here to specify the test list, rather than hard-coding them in at the end with require calls.

@jzaefferer

This comment has been minimized.

Show comment Hide comment
@jzaefferer

jzaefferer Mar 3, 2013

Member

@JamesMGreene would be great if you could put together a new PR that merges this one, #401, runs this new task as part of the default task and fixes the 12 failures.

Regarding the failures: In Rhino etc. there was no setTimeout, so checking for that inside our tests was enough to get them to pass. Node has setTimeout, but no window and document globals. Need to figure out if we just need additional checks for those or if we should replace or enhance the setTimeout checks.

Member

jzaefferer commented Mar 3, 2013

@JamesMGreene would be great if you could put together a new PR that merges this one, #401, runs this new task as part of the default task and fixes the 12 failures.

Regarding the failures: In Rhino etc. there was no setTimeout, so checking for that inside our tests was enough to get them to pass. Node has setTimeout, but no window and document globals. Need to figure out if we just need additional checks for those or if we should replace or enhance the setTimeout checks.

@JamesMGreene

This comment has been minimized.

Show comment Hide comment
@JamesMGreene

JamesMGreene Mar 19, 2013

Member

@twada: You need to sign the CLA before we can work on merging this PR. Please add a comment here when you've done so. Thanks!

Member

JamesMGreene commented Mar 19, 2013

@twada: You need to sign the CLA before we can work on merging this PR. Please add a comment here when you've done so. Thanks!

@twada

This comment has been minimized.

Show comment Hide comment
@twada

twada Mar 19, 2013

@JamesMGreene Thanks! I signed. 👌

twada commented Mar 19, 2013

@JamesMGreene Thanks! I signed. 👌

@JamesMGreene

This comment has been minimized.

Show comment Hide comment
@JamesMGreene

JamesMGreene Jun 4, 2013

Member

@twada: I'm not sure how this worked for you before as I am getting issues immediately upon hooking this up:

Running "test-on-node" task
[DEBUGGING] QUnit:
{ module: [Function],
  asyncTest: [Function],
  test: [Function],
  expect: [Function],
  start: [Function],
  stop: [Function],
  ok: [Function],
  equal: [Function],
  notEqual: [Function],
  propEqual: [Function],
  notPropEqual: [Function],
  deepEqual: [Function],
  notDeepEqual: [Function],
  strictEqual: [Function],
  notStrictEqual: [Function],
  throws: [Function],
  raises: [Function],
  equals: [Function],
  same: [Function],
  begin: [Function],
  done: [Function],
  log: [Function],
  testStart: [Function],
  testDone: [Function],
  moduleStart: [Function],
  moduleDone: [Function]
}
Warning: Cannot set property 'autorun' of undefined Use --force to continue.

Aborted due to warnings.

This is due to the fact that we are only exporting QUnit.constructor.prototype but not QUnit itself as in the browser exports — which provides the other important methods we need for Node support, e.g.: config, load, etc.

Member

JamesMGreene commented Jun 4, 2013

@twada: I'm not sure how this worked for you before as I am getting issues immediately upon hooking this up:

Running "test-on-node" task
[DEBUGGING] QUnit:
{ module: [Function],
  asyncTest: [Function],
  test: [Function],
  expect: [Function],
  start: [Function],
  stop: [Function],
  ok: [Function],
  equal: [Function],
  notEqual: [Function],
  propEqual: [Function],
  notPropEqual: [Function],
  deepEqual: [Function],
  notDeepEqual: [Function],
  strictEqual: [Function],
  notStrictEqual: [Function],
  throws: [Function],
  raises: [Function],
  equals: [Function],
  same: [Function],
  begin: [Function],
  done: [Function],
  log: [Function],
  testStart: [Function],
  testDone: [Function],
  moduleStart: [Function],
  moduleDone: [Function]
}
Warning: Cannot set property 'autorun' of undefined Use --force to continue.

Aborted due to warnings.

This is due to the fact that we are only exporting QUnit.constructor.prototype but not QUnit itself as in the browser exports — which provides the other important methods we need for Node support, e.g.: config, load, etc.

@jzaefferer

This comment has been minimized.

Show comment Hide comment
@jzaefferer

jzaefferer Jun 4, 2013

Member

@JamesMGreene that export looks like a bug introcued here: fb9ddcd#L3L2182 - based on the commit comment and the lack of node-based testing, we should just revert that. Can you take care of that?

Member

jzaefferer commented Jun 4, 2013

@JamesMGreene that export looks like a bug introcued here: fb9ddcd#L3L2182 - based on the commit comment and the lack of node-based testing, we should just revert that. Can you take care of that?

@JamesMGreene

This comment has been minimized.

Show comment Hide comment
@JamesMGreene

JamesMGreene Jun 4, 2013

Member

@jzaefferer: I tried that (seemed like the obvious thing to do) before I posted the comment but it seems like it is still missing some things (unless I'm missing something 😄), e.g. test, asyncTest, module, expect, the lifecycle logging hook functions, etc.:

Running "test-on-node" task
[DEBUGGING] QUnit:
{ constructor: [Function: F],
  urlParams: {},
  isLocal: true,
  assert:
   { ok: [Function],
     equal: [Function],
     notEqual: [Function],
     propEqual: [Function],
     notPropEqual: [Function],
     deepEqual: [Function],
     notDeepEqual: [Function],
     strictEqual: [Function],
     notStrictEqual: [Function],
     throws: [Function] },
  config:
   { queue: [],
     blocking: true,
     hidepassed: false,
     reorder: true,
     altertitle: true,
     requireExpects: false,
     urlConfig: [ [Object], [Object] ],
     modules: {},
     begin: [],
     done: [],
     log: [],
     testStart: [],
     testDone: [],
     moduleStart: [],
     moduleDone: [],
     filter: undefined,
     module: undefined,
     testNumber: null,
     autorun: true },
  init: [Function],
  reset: [Function],
  triggerEvent: [Function],
  is: [Function],
  objectType: [Function],
  push: [Function],
  pushFailure: [Function],
  url: [Function],
  extend: [Function: extend],
  id: [Function: id],
  addEvent: [Function: addEvent],
  addClass: [Function: addClass],
  hasClass: [Function: hasClass],
  removeClass: [Function: removeClass],
  load: [Function],
  equiv: [Function],
  jsDump:
   { parse: [Function],
     typeOf: [Function],
     separator: [Function],
     indent: [Function],
     up: [Function],
     down: [Function],
     setParser: [Function],
     quote: [Function: quote],
     literal: [Function: literal],
     join: [Function: join],
     depth: 1,
     parsers:
      { window: '[Window]',
        document: '[Document]',
        error: [Function],
        unknown: '[Unknown]',
        null: 'null',
        undefined: 'undefined',
        function: [Function],
        array: [Function: array],
        nodelist: [Function: array],
        arguments: [Function: array],
        object: [Function],
        node: [Function],
        functionArgs: [Function],
        key: [Function: quote],
        functionCode: '[code]',
        attribute: [Function: quote],
        string: [Function: quote],
        date: [Function: quote],
        regexp: [Function: literal],
        number: [Function: literal],
        boolean: [Function: literal] },
     HTML: false,
     indentChar: '  ',
     multiline: true },
  diff: [Function]
}
Warning: Object #<F> has no method 'begin' Use --force to continue.

Aborted due to warnings.
Member

JamesMGreene commented Jun 4, 2013

@jzaefferer: I tried that (seemed like the obvious thing to do) before I posted the comment but it seems like it is still missing some things (unless I'm missing something 😄), e.g. test, asyncTest, module, expect, the lifecycle logging hook functions, etc.:

Running "test-on-node" task
[DEBUGGING] QUnit:
{ constructor: [Function: F],
  urlParams: {},
  isLocal: true,
  assert:
   { ok: [Function],
     equal: [Function],
     notEqual: [Function],
     propEqual: [Function],
     notPropEqual: [Function],
     deepEqual: [Function],
     notDeepEqual: [Function],
     strictEqual: [Function],
     notStrictEqual: [Function],
     throws: [Function] },
  config:
   { queue: [],
     blocking: true,
     hidepassed: false,
     reorder: true,
     altertitle: true,
     requireExpects: false,
     urlConfig: [ [Object], [Object] ],
     modules: {},
     begin: [],
     done: [],
     log: [],
     testStart: [],
     testDone: [],
     moduleStart: [],
     moduleDone: [],
     filter: undefined,
     module: undefined,
     testNumber: null,
     autorun: true },
  init: [Function],
  reset: [Function],
  triggerEvent: [Function],
  is: [Function],
  objectType: [Function],
  push: [Function],
  pushFailure: [Function],
  url: [Function],
  extend: [Function: extend],
  id: [Function: id],
  addEvent: [Function: addEvent],
  addClass: [Function: addClass],
  hasClass: [Function: hasClass],
  removeClass: [Function: removeClass],
  load: [Function],
  equiv: [Function],
  jsDump:
   { parse: [Function],
     typeOf: [Function],
     separator: [Function],
     indent: [Function],
     up: [Function],
     down: [Function],
     setParser: [Function],
     quote: [Function: quote],
     literal: [Function: literal],
     join: [Function: join],
     depth: 1,
     parsers:
      { window: '[Window]',
        document: '[Document]',
        error: [Function],
        unknown: '[Unknown]',
        null: 'null',
        undefined: 'undefined',
        function: [Function],
        array: [Function: array],
        nodelist: [Function: array],
        arguments: [Function: array],
        object: [Function],
        node: [Function],
        functionArgs: [Function],
        key: [Function: quote],
        functionCode: '[code]',
        attribute: [Function: quote],
        string: [Function: quote],
        date: [Function: quote],
        regexp: [Function: literal],
        number: [Function: literal],
        boolean: [Function: literal] },
     HTML: false,
     indentChar: '  ',
     multiline: true },
  diff: [Function]
}
Warning: Object #<F> has no method 'begin' Use --force to continue.

Aborted due to warnings.
@JamesMGreene

This comment has been minimized.

Show comment Hide comment
@JamesMGreene

JamesMGreene Jun 4, 2013

Member

P.S. Figured this out... it's because our extend doesn't go into the prototype.

Member

JamesMGreene commented Jun 4, 2013

P.S. Figured this out... it's because our extend doesn't go into the prototype.

@JamesMGreene

This comment has been minimized.

Show comment Hide comment
@JamesMGreene

JamesMGreene Jul 9, 2013

Member

Replaced by PR #458.

Member

JamesMGreene commented Jul 9, 2013

Replaced by PR #458.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment