Auto generate TypeScript definition to allow chaining #884

Merged
merged 14 commits into from Jul 13, 2016

Projects

None yet

5 participants

@ivogabe
Contributor
ivogabe commented May 27, 2016

I've implemented a generator that will create the TS definitions, with full support for chaining. Writing these type definitions by hand would be a lot of work or inaccurate. See the first comment in types/generator.js for details on how these definitions are created.

Fixes #871. Tests for this would be useful too, see #870.

@jamestalmage @SamVerschueren What do you think?

ivogabe added some commits May 27, 2016
@ivogabe ivogabe Generate type definitions that allow chaining 02f0986
@ivogabe ivogabe Add `notRegex` to TypeScript definitions cf9a7b4
@ivogabe ivogabe Change ava to AVA
724f1e4
@novemberborn
Member

Haven't really looked at the generate script, but I think it'd be fine for it to use ES2015 (and thus require Node 6). It's something we'd have to run locally and then check in the results.

@SamVerschueren SamVerschueren and 1 other commented on an outdated diff May 29, 2016
types/generate.js
+// Writing these definitions by hand is hard. Because of chaining,
+// the number of combinations grows fast (2^n). To reduce this number,
+// illegal combinations are filtered out in `verify`.
+// The order of the options is not important. We could generate full
+// definitions for each possible order, but that would give a very big
+// output. Instead, we write an alias for different orders. For instance,
+// `after.cb` is fully written, and `cb.after` is emitted as an alias
+// using `typeof after.cb`.
+
+var path = require('path');
+var fs = require('fs');
+
+var base = fs.readFileSync(path.join(__dirname, 'base.d.ts')).toString();
+
+// All suported function names
+var allParts = ['serial', 'before', 'after', 'skip', 'todo', 'failing', 'only', 'beforeEach', 'afterEach', 'cb', 'always'];
@SamVerschueren
SamVerschueren May 29, 2016 Collaborator

It would be nice if it was exported from within the library https://github.com/avajs/ava/blob/master/lib/runner.js#L11. Then if something changes, the generate script shouldn't be adjusted.

But then again, should libraries export stuff because of "documentation" purposes.

@ivogabe
ivogabe May 29, 2016 Contributor

Sounds like a good idea, it can be really annoying that the type definitions are not up to date.

@SamVerschueren SamVerschueren and 1 other commented on an outdated diff May 29, 2016
types/generate.js
+var base = fs.readFileSync(path.join(__dirname, 'base.d.ts')).toString();
+
+// All suported function names
+var allParts = ['serial', 'before', 'after', 'skip', 'todo', 'failing', 'only', 'beforeEach', 'afterEach', 'cb', 'always'];
+
+var output = base + generatePrefixed([]);
+fs.writeFileSync(path.join(__dirname, 'generated.d.ts'), output);
+
+// Generates type definitions, for the specified prefix
+// The prefix is an array of function names
+function generatePrefixed(prefix) {
+ var output = 'export namespace ' + ['test'].concat(prefix).join('.') + ' {\n';
+
+ var children = '';
+ var empty = true;
+ for (var i = 0; i < allParts.length; i++) {
@SamVerschueren
SamVerschueren May 29, 2016 Collaborator

I would use allParts.forEach() here.

@ivogabe
ivogabe May 29, 2016 Contributor

As @novemberborn suggested, it would be even better to use a for of loop here

@SamVerschueren SamVerschueren and 1 other commented on an outdated diff May 29, 2016
types/generate.js
+ empty = false;
+ if (!isSorted(parts)) {
+ output += '\t' + 'export const ' + part + ': typeof test.' + parts.sort().join('.') + ';\n';
+ continue;
+ } else if (parts.indexOf('todo') !== -1) {
+ output += '\t' + writeFunction(part, 'name: string', 'void');
+ } else {
+ var type = testType(parts);
+ output += '\t' + writeFunction(part, 'name: string, implementation: ' + type);
+ output += '\t' + writeFunction(part, 'implementation: ' + type);
+ }
+ }
+
+ children += generatePrefixed(parts);
+ }
+ if (empty) {
@SamVerschueren
SamVerschueren May 29, 2016 Collaborator

I would set the output variable at the top to an empty string.

if (output === '') {
    return children;
}

return 'export namespace ' + ['test'].concat(prefix).join('.') + ' {\n' + output + children;

And drop the empty variable

@ivogabe
ivogabe May 29, 2016 Contributor

👍

@SamVerschueren SamVerschueren commented on an outdated diff May 29, 2016
types/generate.js
+ var type = testType(parts);
+ output += '\t' + writeFunction(part, 'name: string, implementation: ' + type);
+ output += '\t' + writeFunction(part, 'implementation: ' + type);
+ }
+ }
+
+ children += generatePrefixed(parts);
+ }
+ if (empty) {
+ output = '';
+ } else {
+ output += '}\n';
+ }
+ return output + children;
+}
+function writeFunction(name, args) {
@SamVerschueren
SamVerschueren May 29, 2016 edited Collaborator

Empty line above this one

@SamVerschueren SamVerschueren and 1 other commented on an outdated diff May 29, 2016
types/generate.js
+ children += generatePrefixed(parts);
+ }
+ if (empty) {
+ output = '';
+ } else {
+ output += '}\n';
+ }
+ return output + children;
+}
+function writeFunction(name, args) {
+ return 'export function ' + name + '(' + args + '): void;\n';
+}
+
+function verify(parts, asPrefix) {
+ var has = arrayHas(parts);
+ if (has('only') + has('skip') + has('todo') > 1) return false;
@SamVerschueren
SamVerschueren May 29, 2016 Collaborator

This is quite weird. You add up booleans? Why not has('only') || has('skip') ...?

@ivogabe
ivogabe May 29, 2016 Contributor

This line checks that at least two of them are true, and that's not very easy to do with ||.

@SamVerschueren
SamVerschueren May 30, 2016 Collaborator

Although it works, it feels kinda hacky. I think I'd rather prefer something like

has('only', 'skip', 'todo').filter(Boolean).length

Or something like that.

@ivogabe ivogabe Address PR feedback & linting errors
4cf7bd5
@ivogabe
Contributor
ivogabe commented May 29, 2016

I've addressed most of the comments and fixed the linting issues. Can you take another look?

@SamVerschueren SamVerschueren commented on an outdated diff May 30, 2016
types/generate.js
@@ -0,0 +1,122 @@
+// TypeScript definitions are generated here.
@SamVerschueren
SamVerschueren May 30, 2016 Collaborator

'use strict'

@sindresorhus sindresorhus commented on an outdated diff May 30, 2016
lib/runner.js
@@ -198,3 +198,6 @@ Runner.prototype.run = function (options) {
return Promise.resolve(this.tests.build(this._bail).run()).then(this._buildStats);
};
+
+
+Runner.chainableMethods = chainableMethods.chainableMethods;
@sindresorhus
sindresorhus May 30, 2016 Member

Should be underscored as it's not something we'd like people to use.

@sindresorhus sindresorhus commented on an outdated diff May 30, 2016
types/generate.js
+// Writing these definitions by hand is hard. Because of chaining,
+// the number of combinations grows fast (2^n). To reduce this number,
+// illegal combinations are filtered out in `verify`.
+// The order of the options is not important. We could generate full
+// definitions for each possible order, but that would give a very big
+// output. Instead, we write an alias for different orders. For instance,
+// `after.cb` is fully written, and `cb.after` is emitted as an alias
+// using `typeof after.cb`.
+
+const path = require('path');
+const fs = require('fs');
+const runner = require('../lib/runner');
+
+const arrayHas = parts => part => parts.includes(part);
+
+const base = fs.readFileSync(path.join(__dirname, 'base.d.ts')).toString();
@sindresorhus
sindresorhus May 30, 2016 Member
const base = fs.readFileSync(path.join(__dirname, 'base.d.ts'), 'utf8');
@sindresorhus sindresorhus and 1 other commented on an outdated diff May 30, 2016
types/generate.js
+// Generates type definitions, for the specified prefix
+// The prefix is an array of function names
+function generatePrefixed(prefix) {
+ let output = '';
+ let children = '';
+
+ for (const part of allParts) {
+ const parts = [...prefix, part];
+
+ if (prefix.includes(part) || !verify(parts, true)) {
+ // Function already in prefix or not allowed here
+ continue;
+ }
+
+ // Check that `part` is a valid function name.
+ // `always` is a valid prefix, for instance of `always.after`,
@sindresorhus
sindresorhus May 30, 2016 Member

@jamestalmage I this correct? I thought any of the modifiers could be the function.

@jamestalmage
jamestalmage Jun 2, 2016 Member

Anything can be a function. Mix and match however you want.

always.after and after.always are the same thing.

You could technically do always.always.always.after - it would still be treated the same way.

The only thing disallowed is certain combinations:

Only one of todo, skip, failing should be allowed in the chain.

always only combines with the after(Each) hooks, etc.

Technically, the JavaScript allows any combination (you will never get a "not a function" TypeError) - we just validate the metadata to make sure the combinations make sense.

@sindresorhus
Member

Can you call the file generate.js => make.js instead and add a npm run script for it, like "make-ts": "node types/make.js".

And Travis is failing.

@SamVerschueren
Collaborator

I would make the file executable. './make.js'

@ivogabe ivogabe Address code review
a275089
@jamestalmage jamestalmage commented on the diff Jun 1, 2016
lib/runner.js
@@ -198,3 +198,5 @@ Runner.prototype.run = function (options) {
return Promise.resolve(this.tests.build(this._bail).run()).then(this._buildStats);
};
+
+Runner._chainableMethods = chainableMethods.chainableMethods;
@jamestalmage
jamestalmage Jun 1, 2016 Member

Should we wrap these options in a generator function? So we aren't exporting a mutable copy?

@sindresorhus
sindresorhus Jun 3, 2016 Member

@jamestalmage Doesn't matter. It's for internal use only.

@jamestalmage jamestalmage commented on the diff Jun 1, 2016
types/make.js
@@ -0,0 +1,124 @@
+'use strict';
@jamestalmage
jamestalmage Jun 1, 2016 Member

Shebang line and make it executable?

Does that even work on Windows?

@SamVerschueren
SamVerschueren Jun 2, 2016 Collaborator

👍 on this. I believe it will work as package-cli stuff also works...

@sindresorhus
sindresorhus Jun 3, 2016 Member

Haven't tried, but I don't think it will work. The reason you can use binaries in dependencies is that when npm install binaries it creates a "shim" on Windows that makes it behave like Unix.

@jamestalmage
jamestalmage Jun 3, 2016 Member

Yeah that was my concern. Let's just leave it as is.

@jamestalmage
Member

Are we good here? This all looks good to me. @SamVerschueren - should I merge?

@SamVerschueren
Collaborator

I did not test the output though but generally everything looks good. If you want me to test it more thoroughly, I'm ok with that. If you want to merge, that's ok as well. We can always finetune in follow up commits.

Great work @ivogabe!

@jamestalmage
Member

Should we be storing generated.d.ts in the repo?

Why not add it to .gitignore and create it in a prepublish script?

@SamVerschueren
Collaborator

Good point. Don't see any downside of doing that.

@ivogabe
Contributor
ivogabe commented Jun 4, 2016

I usually don't add generated files to a repo, though I had two reasons now: it's still possible to install AVA from GitHub, and you can easily see what changed when you update the generator script.

@jamestalmage
Member

I'd prefer ignoring it if possible.

It would be awesome if npm had a "installed from GitHub" hook

@SamVerschueren
Collaborator

Can it be executed on postinstall maybe? This would overwrite the generated.d.ts shipped with npm though.

@ivogabe ivogabe Add types/generated.d.ts to gitignore
7a3b4ed
@ivogabe
Contributor
ivogabe commented Jun 8, 2016

I've added it to .gitignore. postinstall won't work, since the generator script requires NodeJS 6. I think that TS users already know that it's usually not possible to install a package from GitHub. Do you agree?

@jamestalmage
Member

I think that TS users already know that it's usually not possible to install a package from GitHub.

I say we start with it this way, and see if there are lots of complaints.

@jamestalmage jamestalmage commented on an outdated diff Jun 13, 2016
"engines": {
"node": ">=0.10.0"
},
"scripts": {
"test": "xo && nyc --cache --reporter=lcov --reporter=text tap --no-cov --timeout=150 test/*.js test/reporters/*.js",
"test-win": "tap --no-cov --reporter=classic --timeout=150 test/*.js test/reporters/*.js",
- "visual": "node test/visual/run-visual-tests.js"
+ "visual": "node test/visual/run-visual-tests.js",
+ "make-ts": "node types/make"
@jamestalmage
jamestalmage Jun 13, 2016 Member

This should probably be a prepublish script. Otherwise I am going to forget this step when publishing.

@jamestalmage
Member

I think that script generation should happen in a prepublish script. Otherwise, I think we are good to go.

@ivogabe
Contributor
ivogabe commented Jun 15, 2016

@jamestalmage I would agree, but I'm afraid that the package cannot be published anymore with that change... The generation script uses ES2015, thus needs Node 6, and npm publish is broken on Node 6.

@SamVerschueren
Collaborator

@ivogabe I can't find features that can only be ran on Node 6 (spread operators for instance). Node 4 supports the majority of those features as well.

@ivogabe ivogabe Add TS generation script as republish script
54a8324
@ivogabe
Contributor
ivogabe commented Jun 15, 2016

That makes life easier then. I've added it in 54a8324.

@jamestalmage
Member

You need to polyfill .includes. I see no reason not to just require('babel-polyfill') for this.

ivogabe added some commits Jun 16, 2016
@ivogabe ivogabe Support Node 4 in TS generation script
984184b
@ivogabe ivogabe Fix linting issue
ef6e73f
@jamestalmage jamestalmage commented on the diff Jun 18, 2016
types/make.js
@@ -0,0 +1,125 @@
+'use strict';
+
+// TypeScript definitions are generated here.
+// AVA allows chaining of function names, like `test.after.cb.always`.
+// The order of these names is not important.
+// Writing these definitions by hand is hard. Because of chaining,
+// the number of combinations grows fast (2^n). To reduce this number,
+// illegal combinations are filtered out in `verify`.
+// The order of the options is not important. We could generate full
+// definitions for each possible order, but that would give a very big
+// output. Instead, we write an alias for different orders. For instance,
+// `after.cb` is fully written, and `cb.after` is emitted as an alias
+// using `typeof after.cb`.
+
+const path = require('path');
@jamestalmage
jamestalmage Jun 18, 2016 Member

const => var throughout. It's not supported on Node 0.12 or 0.10.

Hopefully, that's the last of it. I would like to get this merged.

@sindresorhus
sindresorhus Jun 18, 2016 Member

Same with arrow functions, let, etc, used here. Would be much easier to just require('babel/register') this script.

@sindresorhus
Member
@ivogabe ivogabe Run TS generation script on old NodeJS versions using Babel
43cd1f8
@ivogabe
Contributor
ivogabe commented Jul 1, 2016

Sorry, I've been too busy lately and forgot about this. I've added babel/register, I thought it was easiest to do that in package.json, otherwise I would need to add another file that required babel and the generation script.

@sindresorhus sindresorhus commented on an outdated diff Jul 3, 2016
"engines": {
"node": ">=0.10.0"
},
"scripts": {
"test": "xo && nyc --cache --reporter=lcov --reporter=text tap --no-cov --timeout=150 test/*.js test/reporters/*.js",
"test-win": "tap --no-cov --reporter=classic --timeout=150 test/*.js test/reporters/*.js",
- "visual": "node test/visual/run-visual-tests.js"
+ "visual": "node test/visual/run-visual-tests.js",
+ "prepublish": "npm run make-ts",
+ "make-ts": "node -e \"require('babel-register')({ presets: 'babel-preset-es2015' }); require('./types/make');\""
@sindresorhus
sindresorhus Jul 3, 2016 edited Member

Actually, we could just use babel-node here instead of node. We just need to add babel-cli as a devDependency. Then we could drop this boilerplate.

https://babeljs.io/docs/usage/cli/#babel-node

This will also fix running it on Node.js 0.10 which is currently failing and would let you remove the polyfills in https://github.com/avajs/ava/pull/884/files#diff-148e5d484548912a06d9d0dd7cd4f891R19

ivogabe added some commits Jul 13, 2016
@ivogabe ivogabe Merge remote-tracking branch 'avajs/master' into types-generator 353d104
@ivogabe ivogabe Use babel-node 8008357
@ivogabe ivogabe Accept PromiseLike instead of Promise (#960)
d7c4fcb
@ivogabe
Contributor
ivogabe commented Jul 13, 2016

Does babel-node require more configuration? It still fails on node 0.10 and 0.12. The tests are failing on node 4, has anyone an idea why?

@sindresorhus
Member
sindresorhus commented Jul 13, 2016 edited

@ivogabe Unfortunately yes, since Babel 6 everything needs config.

Replace it with:

"make-ts": "babel-node --presets=babel-preset-es2015 --plugins=transform-runtime types/make.js"
@ivogabe ivogabe Configure babel-node
322445d
@sindresorhus sindresorhus merged commit 1dd6d5b into avajs:master Jul 13, 2016

3 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls Coverage increased (+0.002%) to 96.692%
Details
@sindresorhus
Member

Awesome! Thanks for working on this @ivogabe :)

celebration

@SamVerschueren
Collaborator

Nice work @ivogabe !

@ivogabe
Contributor
ivogabe commented Jul 13, 2016

🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment