Support Typescript incremental build

[glebmachine]

🚀 Feature request

- [ ] new
- [x] build
- [x] serve
- [x] test
- [x] e2e
- [ ] generate
- [ ] add
- [ ] update
- [ ] lint
- [ ] xi18n
- [ ] run
- [ ] config
- [ ] help
- [ ] version
- [ ] doc

Description

Support of Typescript incremental build https://devblogs.microsoft.com/typescript/announcing-typescript-3-4-rc/

[jimbuck]

3.4 is officially out, which is great news! Incremental builds will hopefully save me 10+ minutes each build (I have a set of projects that take 16mins to build all of them).

Out of curiosity may I ask what is currently causing this to be marked as "blocked"?

[clydin]

Angular itself needs to support a TS version before it can be integrated into the tooling.

[RicardoVaranda]

@alan-agius4 @filipesilva Hi Guys! Congrats on the new release! Just wondering if there has been any development with this improvement since TS 3.4 is now supported.

[MartinJohns]

since TS 3.4 is now supported.

@RicardoVaranda Angular 8.0.0 supports TypeScript 3.3, but not 3.4 (which introduced the incremental flag).

https://github.com/angular/angular/blob/master/CHANGELOG.md

add support for TypeScript 3.3

edit: Apparently you can't trust the Angular changelog. Angular 8.0.0 comes with TypeScript 3.4, not with 3.3.

[tomardern]

@MartinJohns Interestingly, 3.4 looks like it was merged into angular-8-beta-12: ef85336

[RicardoVaranda]

Can confirm this, upon doing ng update it actually installed 3.4.2

[MarcosMeli]

Is important to note that 3.4 introduce a regression that make type checking in compilation pain slow in some scenarios.
Now 3.5 is out with this regression fixed: https://devblogs.microsoft.com/typescript/announcing-typescript-3-5/

Speed improvements
TypeScript 3.5 introduces several optimizations around type-checking and incremental builds.
Type-checking speed-ups
Much of the expressivity of our type system comes with a cost – any more work that we expect the compiler to do translates to longer compile times. Unfortunately, as part of a bug fix in TypeScript 3.4 we accidentally introduced a regression that could lead to an explosion in how much work the type-checker did, and in turn, type-checking time. The most-impacted set of users were those using the styled-components library. This regression was serious not just because it led to much higher build times for TypeScript code, but because editor operations for both TypeScript and JavaScript users became unbearably slow.

[filipesilva]

Angular version 8 supports (and requires) TypeScript 3.4.x. I haven't look at this again, but at the time the only problem I saw was #29872.

We have to try and figure out what else (if anything) is preventing the incremental cache from being used and written.

I'm not sure it can be used in the AOT compiler though, so keep that in mind. Maybe it works, maybe it doesn't.

If anyone is interested in having a look at this, here's a good way to get started:

git clone https://github.com/angular/angular-cli
cd angular-cli
yarn

Then open packages/angular_devkit/build_angular/test/browser/works_spec_large.ts and replace it( with fit(. Also remove the afterEach. Now whenever you run yarn test-large, only this test runs. The results for this test will be in a temporary folder in tests/angular_devkit/build_angular.

At this point you'd just have to hunker down and go figure out what the logic for incremental builds in TS is trying to do, and how to work with it. There's not a lot of docs around it. When I was looking into this, I just added a lot of logs to the TS source in node_modules/typescript.

We use a in-memory file system for TS in packages/ngtools/webpack. So even if the cache file is being written, if TS is writing it then it will stay in memory. It would have to be written there somewhere. Check the packages/ngtools/webpack/src/compiler_host.ts for the FS abstraction used by TS. Maybe log some stuff in the writeFile method.

[FDIM]

From what I understand for incremental build to work and actually save time, files have to be emitted to outDir. If the incremental flag is true, there will be an extra tsconfig.tsbuildinfo file that holds hashes of involved files.

When we build with the CLI, outDir does not seem to be used. @filipesilva is this part also using in-memory file system during build? If it is virtualized, do you know if differential build reuses the same virtual FS or a new one is created?

I am quite curious about this, for my little chrome extension (for e2e tests), if I run tsc command and pass project tsconfig, first build is ~15s, 2nd ~5s (at work we might win minutes of build time)

EDIT: Looks like something in public typescript API is blocking effective use of incremental build programmatically. Either way ts-loader needs to support this first: TypeStrong/ts-loader#913 .

EDIT2: at work with angular 8 build alone currently takes 10 minutes (~5min for es2105 and ~5min for es5)

[filipesilva]

@FDIM I think realistically what would happen is that we'd have to allow that file to be loaded between builds.

Maybe to make sure it's loaded it is enough to give it a custom path instead of the outDir default. But at the end of each build it would still need to be persisted to disk with custom logic.

I am not sure how much this would help for differential loading as each build does use different ES targets. At that point it depends on how much TS itself caches for those cases.

We don't use TS loader, mind you. We have our own loader tailored to the Angular AOT compiler. So I don't think ts-loader supporting it blocks us.

[filipesilva]

For reference, the incremental API issue on TS is microsoft/TypeScript#29978.

[FDIM]

My bad regarding ts-loader, I should have realized that when tinkering with the loader in CLI.

I am fairly sure that writing tsconfig.tsbuildinfo file to disk would not be enough, it also needs already built files from last time to avoid compilation. Anyway, I did manage to output build files to disk after running ng build, but tsbuildinfo was not part of the output - not sure why.

EDIT: I've been playing around quite a bit more with this and it seem the issue linked above needs to be handled first.
When compiler plugin invokes ts program to emit files, typescript does not even try to emit build info as it does not have any (see emitBuildInfo in emitSourceFileOrBundle in typescript.js file, program is undefined). If I do this change - empty build info gets written to src/tsconfig.app.tsbuildinfo file.

Anyway, in order to speed up initial build and make use of incremental flag, ng build would have to write compiled typescript files and buildinfo file somewhere outside, so I guess there will be a need for a cache option that would enable extra emits.

The other part that I haven't figured out is how will typescript will know that info file exists, it did not try to access (checking for existence or reading) it via host methods.

[kukjevov]

Typescript already finished their implementation, so it should be possible to try it here :)