Cache support

[Igmat]

As for now cache is disabled. But it seems that it would greatly increase performance, because of omitting of unnecessary transpilations.
We are able to cache next stuff:

1. tsconfig in order to avoid extra disc I/O. It could be easily implemented in transpile-if-ts.ts or in utils.ts. But how to handle its changes during test running? And should we handle them, cause it doesn't seems to be common operation?
2. All other configs could be cached as tsconfig I guess, but it needs further investigation. @kwonoj you probably know this better, any thoughts?
3. sourceMapCache is easily provided by false value of options.emptyCacheBetweenOperations and may be with using of options.hookRequire = true;(but I'm not sure about this option) in index.ts.
4. Providing fileContentsCache could be problematic. On one hand it seems that we are able to easily implement it in our default-retrieve-file-handler.ts, but on other hand it won't be the same thing as sourceMapCache from source-map-support, because the last one isn't exported, so we have no access to it, more of that: it appears in a lot of different places through source-map-support.js, so two caches could be mis-synchronized and produce a lot of mistakes.

And at last, we have to find a way how to test that all this stuff works correctly and may be that it really improves performance. I guess, we could run jest process in our integration test with --watchAll flag and rewrite .ts, .tsx and .json files using fs, but I'm not sure that it is a good solution and even that it would work correctly.

[kwonoj]

I'm not sure about details of this topic, is this about utilize caching behavior of jest or have separate caching mechanism inside of ts-jest orthogonally?

[Igmat]

Unfortunately, as for now I have no idea how to correctly utilize jest caching, but there is a way to utilize (at least partially) cache of source-map-support for source files and implement our own cache for config files.
But probably we have to investigate jest caching too.
@kwonoj if you know how we can use it inside our package, please provide us with your thoughts 😉 . I'll try to find a way to use it too.

[kwonoj]

how we can use it inside our package, please provide us with your thoughts

: I wish there's easy way to use it too. :/ I've looked into jest's implementation if I could use it for coverage processing to avoid --no-cache , unfortunately jest does not exposes it externally to any processors. Implementationwise it stores as file cache into some location based on simple calc (tmp location + jest + version..) so it can be accessed but it'll be undocumented usage and can break anytime.

In my point of view ideal way to achieve caching is use jest as-is (based on assumption if possible...). In most case jest already precaches and does not hit preprocessor if code is already there. Unless writer changes significant amount of code, preprocessing won't cause performance issue in general - this removes needs of config cache in 1, 2. For case of 4, I'm bit doubt about it as reason mentioned above (jest already caches it), it'll be possibly duplicated effort to create file cache.

But once again, this is all based assumption all functionality does not regresses without --no-cache option which busts cache. Primarily coverage processor is directly affected by this, since it needs preprocessor creates in-mem cache for source maps, which doesn't triggered when cache exists.

[Igmat]

@kwonoj I'm not sure but it seems that jest doesn't cache config files (just try to run tests in this project and change any of tsconfig.json files - it won't trigger test rerun). I guess jest doesn't even know about their existence, because they are read directly from tests or processors, so it seems that we have to cache them (if it affects performance of course) and implement 1, 2.

[kwonoj]

but it seems that jest doesn't cache config files

: you're correct, it's not. Maybe my word is not clear - my point is, if preprocessor just rely on jest's caching behavior then it won't hit frequent access to config file even. For example, if user's codebase is ~1000 but only 10 files changes at the moment after transpilation jest will require preprocessor for those 10 files only. Obviously caching config will benefit performance in general, but my point is about high-level gain by achieving that.

[Igmat]

Ok, I understand your point. Anyway we have to create integration tests that check correctness of preprocessor output with caching before doing anything else in this direction.

[kwonoj]

Anyway we have to create integration tests that check correctness of preprocessor output with caching before doing anything else in this direction.

: yes, 100% agreed.