SchemaGenerator optimization

[bywo]

Hi, I'm working with a large TS codebase and I'm looking to optimize schema generation on file changes. So far, I've gotten SchemaGenerator hooked up to Typescript's incremental program watcher so ts.Program regenerations take ~20ms on file change.

Now, the bottleneck is SchemaGenerator.findRootNode(), which loops through every parsed file (including node_modules) looking for the type definition that matches the type name (taking about 600ms for us). I think we can speed this up by allowing the consumer to hint to SchemaGenerator which files to inspect first.

For example, if we know all of our exported types live in the ./types directory, we could tell SchemaGenerator to prioritize looking at files in the ./types directory. If the types we're looking to find don't exist, then continue looking at the rest of the files in the Program.

A few ideas:

• SchemaGenerator could take a fileComparator: (fileA: ts.SourceFIle, fileB: ts.SourceFile) => number function argument in its constructor that sorts the source files on init.
• SchemaGenerator could take a heuristic: (typeName: string, file: ts.SourceFile) => number function argument that scores files on whether the consumer thinks the type will exist there. SchemaGenerator would then look at high-scoring files first.

What do you think? If you like direction, I'm happy to open a PR

[domoritz]

Hi @bywo! Thanks for suggesting these ideas. Performance is a bit of an issue for me as well (currently it takes about a minute to compile a schema for my codebase) so any improvements are very welcome.

I'm not super happy with either of the suggestions as they don't work well for the command line interface. Do you think we could instead take a list of files/directories that should be prioritized?

[bywo]

Ooh that makes sense! Yea maybe we can accept globs from the command line. Any ideas on what to call the command line flag?

[domoritz]

firstFiles, files, start, ...

[domoritz]

We now use caching and schema generation is really fast.