[Proposal] Use the Ava Test runner

[tejasbubane]

Ava is a new javascript test runner that claims to be futuristic. It built-in babel which allows us to write and run tests in ES2017 without the need of any other tool.

Only thing here is that it does not compile the source code, just the tests. As mentioned in ava's README, this can be easily fixed by using babel-register which hooks into ava and transpiles ESNext code on the fly while running tests.

All it takes is this small snippet of code in package.json:

"ava": {
    "require": [
      "babel-register"
    ]
 }

For our use case (of small exercises), this completely eliminates the need for bundling tools like gulp and to maintain the huge gulpfile and pass it along to the users with each exercise. Only thing we need to pass to users is package.json.

To demonstrate what it looks like I ported the first two exercises hello and leap to use ava tests in this repository. hello uses the standard xunit style that ava gives out of the box. While leap uses the BDD style which is provided by ava-spec.

PS: I also took the opportunity to use the latest ES using babel-preset-env which addresses my previous issue #259.

If maintainers agree, I would like to work on this.

[matthewmorgan]

@tejasbubane this looks very cool. Myself I have been thinking about proposing migrating to using Jest, which appears to provide similar benefits.

Both projects appear similar in popularity, I'm not sure there's a compelling case for migrating to one versus the other. My familiarity with Jest comes from its association with React, which is of course enormously popular these days.

I think there is another consideration involved here though: the gulpfile provides an example of what a typical Javascript build pipeline looks like these days, and I think that's invaluable. Not every user will investigate that, but we should consider that aspect as well.

I have also been thinking about proposing changing the build to use Webpack, which has value for the same reasons that using Gulp has, but is more current technology.

@rchavarria what do you think about this? Is it better to have a dead simple exercise environment, or to help expose users to build tools? @tejasbubane what are your thoughts?

[tejasbubane]

Is it better to have a dead simple exercise environment, or to help expose users to build tools? @tejasbubane what are your thoughts?

I am more of a simple environment guy.

Correct me if I am wrong, but build env and frameworks are not the core focus of exercism. We can probably add some resources to build tools and environments in the resources section.

I should probably investigate jest in context of exercism and see if it works the same way.
To be fair, I have become a fan of on-the-fly transpilation (using babel-register) which may be the reason I am biased towards this approach.

[rchavarria]

I haven't heard about Ava, and I've heard about Jest a little bit. So, I have no strong opinion on any of them. But something is clear, build tools are evolving.

For this track, it started using a build tool because of the need of the transpilation step. Nowadays, it seems to be better solved issue.

I don't care too much if we use Ava or Jest, but it seems that any of them will simplify things for final users, and that's a good thing.

We could start thinking about what we want to acomplish, more than the tool we want to use. How would final users use Ava or Jest? Is a complex installation needed? Is it complex to run tests? Sorry if the questions look stupid, but I don't know about Ava or Jest.

[tejasbubane]

I don't care too much if we use Ava or Jest, but it seems that any of them will simplify things for final users, and that's a good thing.

We could start thinking about what we want to acomplish, more than the tool we want to use.

Good point. 👍

How would final users use Ava or Jest? Is a complex installation needed? Is it complex to run tests?

In my case, all a user needs to do is cd into the exercise directory, run npm install and then run npm test. The package.json has ava and babel as devDependencies and a script to run ava.

[rchavarria]

It looks promising. I'm worried about two things:

1. What happens to users with some completed exercises on this track? Your workflow seem identical to the current one (npm install, npm test, well the current one can be run as gulp test).
2. Should test specs be modified? Now, they are based on Jasmine. Does Ava follow the same approach? I mean, describe blocks with it blocks, where it blocks are the spec,...

[matthewmorgan]

I can speak to one thing: Jest uses essentially the same syntax as Jasmine, FWIW. I have so far been able to run existing tests with no modifications, although I could imagine there might be some matchers that might not work out of the box. @tejasbubane how about Ava?

[matthewmorgan]

So @rchavarria @tejasbubane I did some further checking in to this.

1. Ava does not use the same syntax, so the tests would have to be rewritten.
2. Jest can be configured to automatically transpile. This would greatly simplify the directory structure. We could change the devDependencies in package.json to be simply:

  "devDependencies": {
    "babel-jest": "^19.0.0",
    "babel-preset-env": "^1.4.0",
    "jest": "^19.0.2"
  },

and we would need to serve a .babelrc file:

{
  "presets": ["env"]
}

The tests can then simply be run by calling jest from the command line in the exercise directory-- no arguments needed.

That would eliminate the need for the gulp build pipeline for testing. If we wanted to continue to provide linting as an option, we could do that, providing an npm script for doing test, linting, or linting then testing as we wanted. It would be pretty simple to set up.

@tejasbubane I feel like the idea behind making the process simpler for the users is very valuable. My gut is telling me that we'd be better off making this change using Jest than Ava, because it wouldn't require changing the tests themselves and the benefits are equivalent.

If we can get agreement on this I'm happy to do the work or work on it with others. LMK.

[rchavarria]

If Ava uses a different syntax... I wouldn't go that path.

Instead of having a .babelrc file, would it be possible to include it in package.json? I know other tools (such as ESLint) support that. It's just to have less files moving around.

I like the idea of having a linting tool. I hope npm scripts will remain simple.

What do you think @tejasbubane ?

[tejasbubane]

If Ava uses a different syntax... I wouldn't go that path.

Yes Ava uses a different syntax. So if jest works with the given syntax, (which it should since it uses jasmine under the hood) it makes sense to go the Jest way. I am not sure what all config will be required and if it will comsume ES6 code directly without the need of a build tool.

I will put together a small repo with all jest config for couple of exercises like I did for ava.

[matthewmorgan]

@tejasbubane @rchavarria this sounds like a good plan to me. @tejasbubane see my comments above, it was trivial to get it working. If we could get @rchavarria's suggestion about getting the babel config into package.json instead of a separate file, that would be awesome. I played around with it for a couple of minutes without success, but if it's doable that would be preferable-- one less file to serve!

[rchavarria]

I know other tools use the approach of extending the package.json file. Don't spend too much time with that because I don't even know if it's possible or not. I was just speaking my mind, without knowing the possibilities of the tool. Sorry if that confused you 😞

[matthewmorgan]

@tejasbubane I tried to reach you directly via gitter today-- I got so excited about this idea that I have opened a PR. @rchavarria if you would have a look too please, I think this will work. It may resolve a few other issues in the process.