Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deferred execution #362

Closed
balupton opened this issue Apr 4, 2012 · 22 comments
Closed

Deferred execution #362

balupton opened this issue Apr 4, 2012 · 22 comments
Labels

Comments

@balupton
Copy link

@balupton balupton commented Apr 4, 2012

Hey TJ,

Spent the afternoon improving the unit tests for DocPad, but spent about 6 hours tumbling down the rabbit hole of weirdness.

The first issue I came across, is without line 14 in my plugins.test.coffee file, mocha exits absolutely instantly, not good.

The second issue is a bit more complicated, let's now start doing the test, in the same file we hit line 33 which fires another describe for the pluginName, then we continue to call the plugin tester's test function which is all fine and good, however the describes and tests for testLoaded and testGeneration are no longer grouped under the pluginName, although the initial taskCreation is grouped under the pluginName - go figure?!?!

It seems to me that mocha tries to be a lot smarter than it actually is, in terms of predicting when our tests or test suites have actually finished - and it is failing quite miserably in my use cases.

It seems the only solution to this is to create an interface such as:

// Using Total
// tests fire instantly, and when total is hit, then we are done
// the total is set manually at the start by the user
describe('my plugin',function(desc){
    desc.total = 2;
    desc.it('should know 5*5',function(test){
        assert.equal(25,5*5);
        test.done();
    });
    desc.it('should know 5*10',function(test){
        assert.equal(50,5*10);
        test.done();
    });
});

// Using Push
// tests are appended to a queue, and then fired at the end via a sync or async call
// the total is calculated automatically by the queue length
describe('my plugin',function(desc){
    desc.it('should know 5*5',function(test){
        assert.equal(25,5*5);
        test.done();
    });
    desc.it('should know 5*10',function(test){
        assert.equal(50,5*10);
        test.done();
    });
    desc.async();
    // or for sync: desc.sync();
});

that way you always have the correct hierarchy, and will always know exactly when all the children have finished. In DocPad and my other work, we use this paradigm all the time with balUtil's Groups.

What are your thoughts?

Update: Not sure if this will ever make it into Mocha, so I created Joe, which works really really well.

@tj

This comment has been minimized.

Copy link
Contributor

@tj tj commented Apr 4, 2012

ah I see.. you're wanting to meta-generate them from async stuff. I suppose we could have some mechanism to tell mocha to wait, though personally I think setup stuff like that might as well be sync (if possible)

@balupton

This comment has been minimized.

Copy link
Author

@balupton balupton commented Apr 4, 2012

Yeah, not possible to have any of that setup stuff sync, otherwise I surely would!

I've written a simple test for mocha that shows this problem perfectly: https://gist.github.com/2306572

For me there seems to be three issues:

  1. mocha exits early when it reaches async code - solution here will be telling mocha to wait
  2. mocha has no way of identifying which async tests go with which async suites - as the identify process is sync, solution here will be to have some api like:
    • adding suites: mocha.addSuite('suiteName',function(suite){})
    • adding tests to suites: suite.addTest('test name',function(test,done){})
    • adding suites to suites: suite.addSuite('test name',function(suite){})
    • adding suite.runSync() or suite.runAsync() to run the suite's queue of tests and suites in async or sync fashions
    • a test is done once test.done() or done() is called (they are the same function, just exposed in two different ways)
    • a suite is done once all the tests and suites in it are done
    • mocha is done once all the tests and suites in it are done
  3. mocha has no way of identifying whether or not all the async tests or suites have run - solution would be the same as the solution for point 2
@balupton

This comment has been minimized.

Copy link
Author

@balupton balupton commented May 5, 2012

Everytime I see ✔ 0 tests complete (1ms) I feel like killing a kitten. How long must this blasphemy last!

(btw I'm joking, no kittens have been harmed...)

@balupton

This comment has been minimized.

Copy link
Author

@balupton balupton commented Jun 4, 2012

Any news on this? Ideally I'd like to keep using Mocha for DocPad, but as this is a real pain, I'm having to consider moving DocPad and all it's plugins to something else :(

Is there any way to speed this up?

@balupton

This comment has been minimized.

Copy link
Author

@balupton balupton commented Jun 4, 2012

Not sure if this will ever make it into Mocha, so I created Joe, which works really really well.

@tj

This comment has been minimized.

Copy link
Contributor

@tj tj commented Jun 4, 2012

I do plan on having this functionality, when it gets done is another question

@jkp

This comment has been minimized.

Copy link

@jkp jkp commented Jul 15, 2012

+1 on this....I really don't want to have to abandon Mocha for this functionality!

@tarruda

This comment has been minimized.

Copy link

@tarruda tarruda commented Aug 10, 2012

+1. Is there no way to force mocha wait for async tests? I have a web application which runs on mysql, sqlite and postgres, the test cases run flawless on mysql/sqlite, but for some reason are totally skipped on postgres(If I isolate the postgresql tests I also get " ✔ 0 tests complete (1ms) ".

@tj

This comment has been minimized.

Copy link
Contributor

@tj tj commented Aug 10, 2012

@tarruda the db drivers should be queuing, otherwise yeah you need stupid callbacks all over

@tarruda

This comment has been minimized.

Copy link

@tarruda tarruda commented Aug 10, 2012

TJ, I'm not sure if I understand whats causing mocha to early exit the tests. Currently I'm using a simple wrapper on top of database drivers which queues/serializes all statements (https://github.com/tarruda/node-sqli/blob/master/lib/sqli.js#L137). This sqli wrapper uses mocha for its unit tests and it works even for the postgres version.

Perhaps you could give me a hint on which part of mocha's code I need to tweak in order to workaround this issue ? I just need to run a bunch of tests on postgres every once in a while, so its no problem for me adjusting a flag in mocha when I need.

@tj

This comment has been minimized.

Copy link
Contributor

@tj tj commented Aug 10, 2012

we basically just need a flag to tell mocha to defer, then you will manually do mocha.run() at some point

@maraujop

This comment has been minimized.

Copy link

@maraujop maraujop commented Aug 14, 2012

+1 for solving this issue.

@donnut

This comment has been minimized.

Copy link

@donnut donnut commented Oct 2, 2012

+1 from me too. I'm using requirejs, node and mocha. I just learned that the dev2.1 version of requirejs became async. This means mocha is finished before requirejs finished loading. We need a way to defer mocha until requirejs finished loading. I'm not aware of the requirejs release path but this change could be very well included in the next 2.0.7 version and I think more people will be surprised by the '0 tests complete' response.

tarruda added a commit to tarruda/mocha that referenced this issue Jan 17, 2013
This function signals mocha to pause processing files while tests are
generated asynchronously.

For this to work, a few changes were required in the Mocha and Runner classes:

- In the Mocha class, the 'loadFiles' method was refactored to work
  asynchonously.
- In the Runner class, the globals initialization was moved out of the
  constructor into a new method: 'initDefaultGlobals'. This method needs
  to be invoked prior to running the tests, so the runner will take a
  'snapshot' of the global properties.

This commit should also solve issue mochajs#362
@jasonkuhrt

This comment has been minimized.

Copy link

@jasonkuhrt jasonkuhrt commented Aug 12, 2013

+1000

@damassi

This comment has been minimized.

Copy link

@damassi damassi commented Feb 12, 2014

Wow, I can't express how long i had been digging in my code trying to track down this exact same problem. Seriously. No hair left

@humphd

This comment has been minimized.

Copy link

@humphd humphd commented May 16, 2014

I think my issue might be related to this. I have a large set of tests originally written with requirejs for browser-based mocha. I recently wanted to run them in node, and mocha won't wait until requirejs finishes loading them. My work around is this (which allows it to work):

var requirejs = require('requirejs');

describe("Mocha needs one test in order to wait on requirejs tests", function() {
  it('should wait for other tests', function(){
    require('assert').ok(true);
  });
});

requirejs(["tests/test-manifest"]);

I'm then running with mocha --no-exit tests/node-runner.js. Having that single test in the queue, mocha seems to be OK waiting on the async tests to get loaded by requirejs. Took me a long time to stumble on this, though.

@arcanis

This comment has been minimized.

Copy link

@arcanis arcanis commented Jun 22, 2014

Also affected by this issue

@KingScooty

This comment has been minimized.

Copy link

@KingScooty KingScooty commented Jun 27, 2014

+1. Me too.

@adamgruber

This comment has been minimized.

Copy link
Contributor

@adamgruber adamgruber commented Jul 18, 2014

+1. I'm writing a custom reporter that does a bunch of async stuff on 'end' event. Mocha exits early before my code is done.

@rfink

This comment has been minimized.

Copy link

@rfink rfink commented Nov 19, 2014

+1

@boneskull boneskull added the feature label Nov 21, 2014
@boneskull

This comment has been minimized.

Copy link
Member

@boneskull boneskull commented Nov 21, 2014

see #1439

@thecyman

This comment has been minimized.

Copy link

@thecyman thecyman commented Nov 28, 2014

+1
I hope @boneskull 's code changed are merged, but until then, some workarounds.. These won't work for every case mentioned here, and it's a bit ugly, but..
You can use the done() to allow async tests: http://lostechies.com/derickbailey/2012/08/17/asynchronous-unit-tests-with-mocha-promises-and-winjs/

describe("a test", function(){
  var foo = false;
  beforeEach(function(done){
    setTimeout(function(){
      foo = true;
      // complete the async beforeEach
      done();
    }, 50);
  });

In worst-case scenario, you can use sleep or, for very long test cases where manual entry is okay, perhaps readline-sync.
https://www.npmjs.org/package/sleep
https://www.npmjs.org/package/readline-sync

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.