Reconsider deprecating planned assertions argument in test.done() #744

Open
philmander opened this Issue Dec 4, 2013 · 4 comments

3 participants

@philmander

In my Casper test suite, each test case is modelling a particular user journey within my webapp. Each user journey may replicate common parts, so to avoid code duplication, I have a setup where common tests are imported as modules. Here's a simple example, logging in to the app at the start of a test case:

var login = require("../modules/login");
casper.test.begin('Make a payment', function(test) {

    casper.start();

    casper.then(function() {
        login.loginAsUser("lisa", "password");
    });
    casper.then(function() {
        //do some other stuff
    });

    casper.run(function() {
        test.done();
    });
});

This seems to be good practice to me. My problem is that it's difficult to know the number of planned assertions in this code, since it is made up of many imported functions which each have their own assertions. And changing the number of assertions in a module will break the tests cases downstream.

My solution is to return the number of planned assertions from each imported test function and accumulate them, then pass the planned total to done(), like this:

var login = require("../modules/login");
casper.test.begin('Make a payment', function(test) {

        //just one assertion
    var total = 1;
    test.pass("blah");

    casper.start();

    casper.then(function() {
        //more assertions happen in here:
        total += login.loginAsUser("lisa", "password");
    });
    casper.then(function() {
        //do some other stuff
    });

    casper.run(function() {
        test.done(total);
    });
});

However, as passing the planned assertions to test.done is now deprecated it means that, obviously, this won't be possible in the future. Is it possible to un-deprecate this feature or is there, perhaps, a more obvious solution to my problem?

@n1k0
CasperJS member

Very good point. I must admit I never considered the case of modularized asserters… which sounds so obvious now I think about it that I feel dumb.

The problem with passing the expected number of tests executed to done() is that if something fail roughly, that call is never performed at all, hence why we declare it even before the test execution starts.

The only alternative I could see atm is code parsing/introspection to count assert* calls, which is so weird & ugly I can't even face the fact I've thought about it.

I'll be thinking of this issue for the next few days, in the meanwhile if you have any idea…

@philmander

Not quite sure how I feel about this, but I've evolved my example to look like this;

var login = require("../modules/login");
var planned = 1;
planned += login.loginAsUser.planned;
casper.test.begin('Make a payment', planned, function(test) {

    test.pass("blah");

    casper.start();

    casper.then(function() {
        //more assertions happen in here:
        login.loginAsUser("lisa", "password");
    });
    casper.then(function() {
        //do some other stuff
    });

    casper.run(function() {
        test.done();
    });
});

...using the convention of having a planned property that must be maintained on the imported test functions.

@n1k0
CasperJS member

Nice trick indeed :)

@istr istr added the Documentation label Jan 20, 2016
@istr

We should mention this trick along with the deprecation notice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment