Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Issue #432 - Using a setTimeout stub can stop test suite from continuing #433

Closed
wants to merge 6 commits into from

4 participants

David Vollbracht Jörn Zaefferer Timo Tijhof James M. Greene
David Vollbracht

If you replace the global setTimeout function with a stub (like sinon's useFakeTimers), it can permanently stop the test suite.

When QUnit stops processing the queue to let the browser update, it calls the global setTimeout to schedule the next block of queue processing. If you are unlucky the test currently running may have stubbed setTimeout and its teardown step could still be on queue waiting to run. In this case, QUnit will end up using the stubbed setTimeout instead of the real one, and the entire test suite will stop running.

This change fixes the problem by capturing window.setTimeout when QUnit is loaded and using the captured version rather than referring to window.setTimeout for each call.

Jörn Zaefferer
Owner

Thanks for the contribution! Could you please sign our CLA? http://contribute.jquery.org/CLA/

David Vollbracht

My pleasure. I have signed the CLA.

Jörn Zaefferer
Owner

Finally got a chance again to look at this. The test doesn't pass in Firefox. Could you take a look at that, and rebase the PR against master? Thanks.

test/test.js
((19 lines not shown))
+ ' setup: function() {' +
+ ' this.setTimeout = window.setTimeout;' +
+ ' window.setTimeout = function() {};' +
+ ' },' +
+ ' teardown: function() {' +
+ ' window.setTimeout = this.setTimeout;' +
+ ' }' +
+ ' });' +
+ ' test("just a test", function() { ok(true); });' +
+ ' test("just a test", function() { ok(true); });' +
+ ' </script>' +
+ ' </body>' +
+ ' </html>';
+
+ var frame = document.createElement('iframe');
+ var supportsSrcDoc = !!('srcdoc' in frame);
Jörn Zaefferer Owner

The in test already produces a boolean, no need for the explicit cast.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
David Vollbracht

Sorry - was sick last week and didn't get to it. I'm happy to rebase and fix whatever is happening in firefox this week.

David Vollbracht

I have fixed the firefox issue (innerText, DOH!) and rebased on latest master. Let me know if there's anything else you'd like me to do.

Jörn Zaefferer
Owner

Thanks, landed!

Timo Tijhof Krinkle referenced this pull request from a commit
Timo Tijhof Krinkle Revert "Fixes #432 - Using a setTimeout stub can stop test suite from…
… continuing - closes gh-433"

This reverts commit cf41077.
b56ea44
Timo Tijhof Krinkle reopened this
Timo Tijhof
Collaborator

This fails in IE8, IE7 and IE6:

Died on test #1 undefined: Object doesn't support this property or method 

http://swarm.jquery.org/job/553

James M. Greene
Collaborator

Probably the Date.now() calls. They should be changed to +new Date() or (new Date()).getTime().

Timo Tijhof
Collaborator

Also making use of this oppertunity to point out:

  • Code does not conform to the style guide
  • Does it need the whole iframe thing? Shouldn't it be possible as a regular module with setup/teardown? I suspect the QUnit.config setting is why it is in a separate test suite, however in that case please add a regular test suite in an html file instead of re-inventing QUnit Composite locally.
David Vollbracht

If i create a separate html file for its own test suite, the failure mode of that test suite will be to hang forever without actually reporting failure. Is that ok?

David Vollbracht

I don't have access to IE at the moment, but I will make the date change. Is there a way for me to run the test swarm from my local copy?

David Vollbracht

I'm happy to digest the style guide and ensure the code conforms to it. I'll starting looking at it tomorrow morning.

Timo Tijhof
Collaborator

@qxjit Not TestSwarm, no. But the logic TestSwarm and Jenkins perform is only 1% of the magic. Pretty much everything is the repository itself.

You can open test/index.html of this repository in your browser and run it. Simple as that.


  • Checkout jquery/qunit.git (be sure to include submodules and run npm install).
  • Open http://localhost/jquery/qunit/test (or wherever you put it within your web server doc root) in Internet Explorer. If you (like most of us) don't run Windows and don't have Internet Explorer locally, you can open the URL in either using your own VM or a cloud service that provides a VM. I'd recommend the latter.
    A few cloud services that feature real Internet Explorer access (e.g. not IETester or MultipleIE):
    • SauceLabs (has free/opensource option)
    • BrowserStack

Both of these have easy-to-use and built-in support to set up a tunnel in your browser to forward your localhost, and they have in-browser (using HTML/javascript) viewport of the remote VM that is instantly spawned for you.

After initial set up it takes less than 10 seconds for me to point my web browser (e.g. Chrome) to BrowserStack, paste my local url, pick OS/browser/version, start.

David Vollbracht

I've moved the tests to their suite and replaced spaces with tabs. I think that brings the code into conformance with the style guide. The only thing I'm not sure about is how to integrate the new suite into test swarm. I assume that's something that needs to done.

David Vollbracht

I believe this now conforms to the style guide, and I've separated the tests to their own suite. Does anything need to be done to make it work with the TestSwarm build?

Jörn Zaefferer jzaefferer closed this pull request from a commit
David Vollbracht qxjit Use a local setTimeout reference, add separate unit test suite for th…
…at. Fixes #432 - Using a setTimeout stub can stop test suite from continuing. Closes gh-433
4e62b4a
Jörn Zaefferer jzaefferer closed this in 4e62b4a
Timo Tijhof Krinkle commented on the diff
test/setTimeout.html
@@ -0,0 +1,15 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="UTF-8">
+ <title>QUnit Fake setTimeout Test Suite</title>
+ <link rel="stylesheet" href="../qunit/qunit.css">
+ <script src="test.js"></script>
+ <script src="deepEqual.js"></script>
+ </head>
+ <body>
+ <div id="qunit"></div>
+ <script src="../qunit/qunit.js"></script>
+ <script src="setTimeout.js"></script>
Timo Tijhof Collaborator
Krinkle added a note

This also needs swarminject.js.

Timo Tijhof Collaborator
Krinkle added a note

Fixed in 65d27d2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Jörn Zaefferer
Owner

Thanks, landed! I squashed, removed the extra scripts (test.js, deepEqual.js), added swarminject (in 65d27d2) and a bit of spacing in the JS file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on May 14, 2013
  1. David Vollbracht
  2. David Vollbracht

    Fix setTimeout stub test in Firefox.

    qxjit authored
    Use innerHTML instead of innerText, since latter
    isn't available in Firefox.
  3. David Vollbracht
  4. David Vollbracht
Commits on May 16, 2013
  1. David Vollbracht
Commits on May 21, 2013
  1. David Vollbracht

    Separate setTimeout tests into js file.

    qxjit authored
    Change spaces to tabs in js and html test files.
This page is out of date. Refresh to see the latest.
3  Gruntfile.js
View
@@ -11,7 +11,8 @@ grunt.initConfig({
qunit: [
"test/index.html",
"test/async.html",
- "test/logs.html"
+ "test/logs.html",
+ "test/setTimeout.html"
]
},
jshint: {
7 qunit/qunit.js
View
@@ -20,6 +20,7 @@ var QUnit,
hasOwn = Object.prototype.hasOwnProperty,
// Keep a local reference to Date (GH-283)
Date = window.Date,
+ setTimeout = window.setTimeout,
defined = {
setTimeout: typeof window.setTimeout !== "undefined",
sessionStorage: (function() {
@@ -466,7 +467,7 @@ QUnit = {
}
// A slight delay, to avoid any current callbacks
if ( defined.setTimeout ) {
- window.setTimeout(function() {
+ setTimeout(function() {
if ( config.semaphore > 0 ) {
return;
}
@@ -489,7 +490,7 @@ QUnit = {
if ( config.testTimeout && defined.setTimeout ) {
clearTimeout( config.timeout );
- config.timeout = window.setTimeout(function() {
+ config.timeout = setTimeout(function() {
QUnit.ok( false, "Test timed out" );
config.semaphore = 1;
QUnit.start();
@@ -1445,7 +1446,7 @@ function process( last ) {
if ( !defined.setTimeout || config.updateRate <= 0 || ( ( new Date().getTime() - start ) < config.updateRate ) ) {
config.queue.shift()();
} else {
- window.setTimeout( next, 13 );
+ setTimeout( next, 13 );
break;
}
}
15 test/setTimeout.html
View
@@ -0,0 +1,15 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="UTF-8">
+ <title>QUnit Fake setTimeout Test Suite</title>
+ <link rel="stylesheet" href="../qunit/qunit.css">
+ <script src="test.js"></script>
+ <script src="deepEqual.js"></script>
+ </head>
+ <body>
+ <div id="qunit"></div>
+ <script src="../qunit/qunit.js"></script>
+ <script src="setTimeout.js"></script>
Timo Tijhof Collaborator
Krinkle added a note

This also needs swarminject.js.

Timo Tijhof Collaborator
Krinkle added a note

Fixed in 65d27d2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ </body>
+</html>
15 test/setTimeout.js
View
@@ -0,0 +1,15 @@
+QUnit.config.updateRate = 1;
+
+module("Module that mucks with time", {
+ setup: function() {
+ this.setTimeout = window.setTimeout;
+ window.setTimeout = function() {};
+ },
+
+ teardown: function() {
+ window.setTimeout = this.setTimeout;
+ }
+});
+
+test("just a test", function() { ok(true); });
+test("just a test", function() { ok(true); });
Something went wrong with that request. Please try again.