Improve tests feedback, linked to #104

[jfmengels]

This is related to #104, and an attempt to improve the RuleTester's feedback. By default, it uses Mocha's describe and it to create rules, but it they are not defined, it uses the functions here.

I've managed to override the RuleTester test creation a bit to use AVA's test rather than Mocha's it, and here are the results:

Before

When run with Mocha

When run with AVA (with --verbose)

When run with AVA and tap-mocha-reporter

The odd test.xxx(t => {}) kind of everywhere are the second part of the title (since it takes as title the code, and the code has a \n in the middle). If we remove the \n, it gets much clearer

Please check out eslint-ava-rule-tester, which exports a modified RuleTester. I'm up for modifying the name, but it's already published (could not make it all work with npm link), which contains the same code that you can see in test/helpers/ava-rule-tester.js (I kept it ll so that you guys could easily see what I did here).

Let me know what you guys think.

PS: Obviously this is just a draft, I haven't migrated all the tests and have even left an error so that you can see the error feedback more easily.

[sindresorhus]

Nice! Looks so much better now :)

[sindresorhus]

Maybe we should do a PR to ESLint adding a hook to their rule checker so any test framework could easily hook into it?

[jfmengels]

Nice! Looks so much better now :)

Yeah!

Maybe we should do a PR to ESLint adding a hook to their rule checker so any test framework could easily hook into it?

Sure. By their contribution guidelines, we should first create an issue though. Not sure I see what the hook should look like though?

[jfmengels]

By the way: Found this comment eslint/eslint#5040 (comment), where they talk about eventually using AVA under the hood for RuleTester.

[jfmengels]

Created an issue in ESLint about this eslint/eslint#6227

[jfmengels]

Now that we know that ESLint considers this okay and describe and it as part of their API (meaning it will be enforced by semver), shall I go ahead with finishing this? If so, I'll go ahead and do eslint-plugin-xo too (and my own plugins).

[jfmengels]

Before

After

Biggest number of test additions you'll ever see in one commit :D
(Lots of diff, but it's mostly just de-indenting all the tests)

[jfmengels]

Hmm, looking into the tests a bit later today (worked on my end before I reinstalled all node_modules). Stay tuned.

[sindresorhus]

shall I go ahead with finishing this?

Let's do it! :D

[jfmengels]

There you go :)

[sindresorhus]

Yay! So much better output :)

[jfmengels]

:)

Will do a PR for XO soon-ish.

[jfmengels]

PS: eslint/eslint#6331, if you have comments to improve this ;)