How to test an ES7 async function

[krnlde]

Hey guys,
I use ES7 async functions and tried to debug them. Having this method to test:

// ...
const local = new WeakMap();

export default class User {

  // ...

  async password(password) {
    if (!password) return local.get(this).get('hash'); // remove this for security reasons!
    if (password.length < 6) throw new Error('New password must be at least 6 characters long');
    if (!password.match(passwordPattern)) throw new Error(`New password must match ${passwordPattern}`);
    local.get(this).set('hash', await Password.hash(password));
  }

  // ...

}

and this test case:

import chai from 'chai';
import chaiAsPromised from 'chai-as-promised';

import User from '../../../server/User';

chai.use(chaiAsPromised);
const expect = chai.expect;

describe('.password()', () => {

  const testuser = new User({username: 'Testuser', password: '123abc'});

  // FINDME
  it(`should throw when too short`, () => {
    return expect(testuser.password('1a')).to.eventually.throw();
  });

  // ...

});

...the test case does not catch the error thrown - instead the case succeeds first and fails later (asynchronously) with an uncaught error because the method threw in the scope of the it() function not the expect().

Any suggestions or advice?

Thanks in advance!

P.S.: Also I created a stackoverflow issue for this a few days ago, but now answer so far. That's why I am calling you guys. http://stackoverflow.com/questions/29334775/how-to-test-an-es7-async-function-using-mocha-chai-chai-as-promised

[krnlde]

btw. expect(() => testuser.password('1a')) and expect(async () => await testuser.password('1a')) do not work either.

The only thing that seems to work right now is catching the error via try catch:

it(`should throw when too short`, async (done) => {
  try {
    await testuser.password('1a');
    done('failed');
  } catch (e) {
    done();
  }
});

But this is ugly and doesn't even use the bdd methods or profit from chai's fluent pattern.

[keithamus]

@krnlde I can't see any reason why that code wouldn't work.

What happens if you change the test to the following?

  it(`should throw when too short`, (done) => {
    testuser.password('1a').then(done, done);
  });

You should see the test fail appropriately with the error, and not timeout. If it times out - its likely that your code isn't working like it should. It it fails with an error, then it is likely something wrong is happening within either chai or chai-as-promised.

[krnlde]

Same issue as mentioned before. The error is not thrown into the Promise, but in the it() scope.

  User
    .password()
      1) should throw when too short


  0 passing (13ms)
  1 failing

  1) User .password() should throw when too short:
     Error: New password must be at least 6 characters long
      at User.callee$1$0$ ...

The 1) at the tail is too late. It shouldn't appear at all. If a normal function is caught by .should.throw or should.eventually.throw no additional Error is shown.

[keithamus]

The test I wrote should error as expected above. The log you pasted looks right to me.

You could try the following:

  it(`should throw when too short`, () => {
    return testuser.password('1a').then(() => throw Error('should error'), () => {})
  });

This should pass the test suite without error. If it does then we know mocha and babel aren't the problem.

[krnlde]

When copy pasting your code I get a syntax error at throw with SyntaxError: ...: Unexpected token.

If I wrap the error in curlies...

  it(`should throw when too short`, () => {
    return testuser.password('1a').then(() => {throw Error('should error')}, () => {})
  });

...the promise never settles and mocha skips after the timeout default of 2000ms.

If I do:

it.only(`should throw when too short`, (done) => {
  testuser.password('1a').then(function () {
    console.log('fullfilled');
    done();
  }, function () {
    console.log('rejected');
    done();
  });
});

The console.log('rejected'); is called properly and the test succeeds.

[krnlde]

More investigation done. Facebook's regenerator runtime definitely returns a new Promise when calling an async function as stated here: https://github.com/facebook/regenerator/blob/master/runtime.js#L106 . I've debugged the whole runtime and can confirm that.
Also the async spec says a Promise should be returned. Here https://github.com/lukehoban/ecmascript-asyncawait#details and here http://wiki.ecmascript.org/doku.php?id=strawman:async_functions#details (same text)

So from my understanding - since we have a Promise that is returned - these assertions both should work:

// expect-style
it.only(`should throw when too short`, () => {
  return expect(testuser.password('1a')).eventually.throw();
});

// or in should-style
it.only(`should throw when too short`, () => {
  return testuser.password('1a').should.eventually.throw();
});

But they don't.

Maybe it has sth to do with this line here https://github.com/facebook/regenerator/blob/master/runtime.js#L122
Where an error is already caught. Afaik errors, once caught, are not available to the promise chain anymore. So what probably happens here is that the callThrow method (which equals Generator.prototype.throw) throws the error in the it() scope. That would explain why a try catch works but an expect(...) doesn't.

Any thoughts?

[keithamus]

What happens if you try the following:

  it(`should throw when too short`, (done) => {
    testuser.password('1a').should.eventually.throw().notify(done);
  });

Also what versions of all software are you using? babel, chai, mocha, chai-as-promised, everything.

[krnlde]

I get

   .password()
     1) should throw when too short


 0 passing (0ms)
 1 failing

 1) User .password() should throw when too short:
    TypeError: Cannot read property 'eventually' of undefined

Your code transformed in the expect-style (expect(testuser.password('1a')).eventually.throw().notify(done);) returns

    .password()
      1) should throw when too short


  0 passing (0ms)
  1 failing

  1) User .password() should throw when too short:
     Error: New password must be at least 6 characters long

as always.

What I'm curious about is that

  testuser.password('1a')
    .then(() => console.log("Shouldn't appear"))
    .catch(() => console.log("All good"))

resolves to All good as expected.

An extract of my package.json (npm installd daily) is:

  "devDependencies": {
    "babel": "latest",
    "chai": "latest",
    "chai-as-promised": "latest",
    "gulp": "latest",
    "gulp-eslint": "latest",
    "gulp-less": "latest",
    "gulp-open": "latest",
    "gulp-plumber": "latest",
    "gulp-run": "latest",
    "gulp-sourcemaps": "latest",
    "mocha": "latest",
    "run-sequence": "latest"
  }

Which resolves to: babel@5.0.8, chai@2.2.0, chai-as-promised@4.3.0 and mocha@2.2.1 at the time of writing this.

I'm running io.js v1.6.3 on a Win8.1 Pro x64.

[krnlde]

I built a runnable.com instance with the test case so you can see what I see http://runnable.com/VSAUjLYcbqoxDfiY

[krnlde]

I got the solution! Expecting async function to actually "throw" is wrong, since it returns a promise. You'll have to expect().to.be.rejected instead.

Here's the modified test case:

it.only(`should be rejected when too short`, () => {
  return expect(testuser.password('1a')).to.be.rejectedWith(Error);
});

// should-style
it.only(`should be rejected when too short`, () => {
  return testuser.password('1a').should.be.rejectedWith(Error);
});

The question is, does an .eventually.throw() make sense? Is it different to .rejectedWith(Error)?

[keithamus]

Oh, duh. Of course! Glad you found the issue @krnlde

.eventually.throw may make sense, especially as an async function throw is a rejection. You should raise this in chai-as-promised as that would be the correct place to file it.

[krnlde]

Done. Thanks for your help @keithamus !