Crazy checker.ts refactor experiment (25kloc)

[Busyrev]

I have made tool-assisted refactoring experiment on typeChecker. The goal is to convert checker.ts from incapsulated js scope to a class. For better extensibility and publicity (#17680).

All tests are passing, excluding linting (because of auto converting). We can fix linting issues later.

How I refactored 25k lines of code:

1. All types moved to ts namespace out of createTypeChecker;
2. Class TypeCheckerImpl was created.
3. All variables from createTypeChecker moved to public properties of TypeCheckerImpl;
4. Property initializers were moved to constructor. (because of constructor parameters dependency)
   declarations now looks ugly, because

   public propName = (false as true) && this.someMethod();

   is only way to auto infer type of function return value; We can fix it later.
5. All nested functions from createTypeChecker converted to public methods of TypeCheckerImpl;
6. For all converted functions auto inserted "this" keyword.
7. For methods that contain nested functions I have to create ugly conv_self to enclosure "this".
8. Added ugly code to bind all methods. We can fix it later.
9. Now TypeCheckerImpl instance is not exposed to public, but I want to do it later.

Here is commit in my fork:
Busyrev@697d90e
Unfortunetly GitHub can`t show diff, and that is one more point for necessity of refactoring.

Raw checker.ts before

https://raw.githubusercontent.com/Microsoft/TypeScript/master/src/compiler/checker.ts

after

https://raw.githubusercontent.com/Busyrev/TypeScript/checker-class-public-experimental/src/compiler/checker.ts

This is only experiment.

I have spend around 16 hours for these transforms. I think that this experiment is successful.
For better typescript future I think checker.ts should be separated into multiple files/classes, and this is the first step. This code is not merge ready. Of course we should think about methods to be provided to public api, and may be unsafe access for people who do really complicated transformations.

What do you think about it?

I can dedicate more time, improve documents and publish tools I've made, if more people support this experiment.

[RyanCavanaugh]

😱 wow!

Did you see any change in performance?

[Busyrev]

@RyanCavanaugh I think no. I didn't write performance tests, but all regular typescript tests run in same time as usual build.

[Busyrev]

got an idea how to prevent splitting property declarations and initialisation, using

constructor(public host: TypeCheckerHost, public produceDiagnostics: boolean )

It fixes problem of properties dependency. I`ll try it

[kitsonk]

@Busyrev as built compiler supports tsc --diagnostics and give you some level of indication of performance, if you build something substantial with the two versions.

[ghost]

I ran our internal performance tests and it looks like there is a large increase in check time. (There were also large increases in emit time, which may just be my hard drive acting up.) There was no change to parse or bind time or memory usage.

Monaco - node (v8.2.1, x64)

Project	Baseline	Current	Delta	Best	Worst
Memory used	357,006k (± 0.00%)	356,252k (± 0.00%)	-754k (- 0.21%)	356,247k	356,258k
Parse Time	2.10s (± 1.10%)	2.10s (± 0.62%)	+0.01s (+ 0.43%)	2.09s	2.12s
Bind Time	0.81s (± 1.15%)	0.81s (± 1.61%)	+0.01s (+ 0.74%)	0.80s	0.83s
Check Time	4.19s (± 0.60%)	4.89s (± 0.53%)	+0.70s (+ 16.62%)	4.86s	4.92s
Emit Time	2.77s (± 0.73%)	6.92s (± 1.01%)	+4.15s (+149.96%)	6.88s	7.03s
Total Time	9.87s (± 0.45%)	14.74s (± 0.48%)	+4.87s (+ 49.37%)	14.69s	14.84s

Monaco - tsc (x86)

Project	Baseline	Current	Delta	Best	Worst
Parse Time	1.51s (± 3.64%)	1.48s (± 0.88%)	-0.03s (- 1.85%)	1.47s	1.50s
Bind Time	0.58s (± 1.82%)	0.58s (± 2.95%)	-0.01s (- 1.03%)	0.56s	0.60s
Check Time	3.98s (± 0.98%)	4.93s (± 0.58%)	+0.96s (+ 24.13%)	4.89s	4.95s
Emit Time	7.54s (± 6.52%)	7.24s (± 0.41%)	-0.30s (- 3.98%)	7.23s	7.29s
Total Time	13.61s (± 3.93%)	14.24s (± 0.43%)	+0.63s (+ 4.64%)	14.16s	14.31s

TFS - node (v8.2.1, x64)

Project	Baseline	Current	Delta	Best	Worst
Memory used	309,696k (± 0.01%)	308,820k (± 0.01%)	-876k (- 0.28%)	308,805k	308,841k
Parse Time	1.53s (± 1.92%)	1.54s (± 4.00%)	+0.01s (+ 0.85%)	1.45s	1.58s
Bind Time	0.68s (± 4.95%)	0.69s (± 10.07%)	+0.01s (+ 1.02%)	0.66s	0.80s
Check Time	3.60s (± 0.51%)	4.24s (± 0.79%)	+0.64s (+ 17.84%)	4.20s	4.28s
Emit Time	2.45s (± 0.33%)	4.09s (± 1.23%)	+1.65s (+ 67.31%)	4.06s	4.17s
Total Time	8.26s (± 0.33%)	10.57s (± 0.66%)	+2.31s (+ 27.92%)	10.51s	10.66s

TFS - tsc (x86)

Project	Baseline	Current	Delta	Best	Worst
Parse Time	1.09s (± 5.00%)	1.07s (± 0.90%)	-0.02s (- 1.84%)	1.06s	1.08s
Bind Time	0.58s (± 5.24%)	0.57s (± 3.35%)	-0.00s (- 0.86%)	0.55s	0.59s
Check Time	3.26s (± 1.15%)	3.95s (± 1.10%)	+0.70s (+ 21.44%)	3.90s	3.99s
Emit Time	4.44s (± 1.49%)	4.36s (± 1.21%)	-0.08s (- 1.85%)	4.29s	4.41s
Total Time	9.37s (± 1.10%)	9.96s (± 0.86%)	+0.59s (+ 6.32%)	9.89s	10.05s

I'd really like to see the checker cut down to size -- it's so big that vscode doesn't support collapsing code, and it takes several seconds to update errors. I think some of the candidates for removal from checker.ts were isTypeAssignableTo and typeToString. @sandersn might have some comments.

[Busyrev]

@andy-ms thanks for tests! I believe we can beat this 25% slowdown.
Now all methods are bound. I'm working on auto binder now, only for methods used not for calling. It can increase speed, may be. And I`m thinking about removing method createTypeChecker from implementation, and implement Typechecker interface directly.

[ghost]

Hi @Busyrev , sorry for the delay. It looks like your branch Busyrev/TypeScript/checker-class-public-experimental is still on the Aug 17 commit "First ugly try to refactor checker.ts". I can run performance tests again if you push to that branch.

[chicoxyzzy]

That'll be much easier to refactor incrementally if checker.ts would be splitted in modules but it seems there is chicken egg problem here :(

[kevinbarabash]

An incremental that could be used is to move some of the variables that createTypeChecker defines near the top into a "context" object and then pass that context object to the functions that use those variables. You could start with those variables that are least used. Once a function no longer makes uses of any variables from the closure it can be extracted.

It might even make sense to have a couple of objects. I could very easily seen all of the type intrinsics defined grouped together on their own object. I see that createTypeChecker is called multiple times in program.ts. Is there any worry about reusing types created by createType across multiple calls to createTypeChecker?

[Timmmm]

Can someone explain the performance issues? It seems crazy to me that you can't split up a 41k line file because of that. It makes editing it very difficult - not only the performance issues, but you can't even jump to errors or changes using the scrollbar.

[Busyrev]

@Timmmm I think that there is two major performanse issues:

1. all methods are binded. This may be fixed.
2. using members instead of local variables is slightly slower. And I know no way to speed it up.

[RyanCavanaugh]

This gets asked a ton of times, so here's the short version

Imagine you have this code structure

function fn(x) {
  // There are ~450 of these variables
  let y1 = someFunc1(x);
  let y2 = someFunc2(x);

  // There are ~1200 of these functions
  function a() {
    console.log(y1 - x);
  }
  function b(z) {
    console.log(y2 + x * z);
  }
}

It is a fact in modern JS engines that bare-name lookups like y1 and y2 are much faster than property accesses like obj.y1 and obj.y2. The challenge, then: how do you "split up" fn in a way that preserves that performance improvement?

[fbartho]

I don't see discussion of this topic in this PR, but is the a reason you couldn't use a bundler step to ensure production users aren't slowed down?

A sufficiently smart bundler could properly inline each of those functions to create the same "output" we have today? -- Preserving the performance wins.

Additionally: in your example, y1, y2, … are effectively global variables from the perspective of a, b, … Proposal: instead of scope-capturing, they could be explicit, typed parameters. Doing that would make it easier to slice a, b, … out into separate files/modules for ease of developer editing & code organization etc. Again, the custom bundler could inline them.

Obviously deciding on the right module boundaries is hard, but it feels like checker.ts is creaking under it's own weight and needs that anyways.

UPDATE: Whoops, brain-fart. I ran across this from twitter, and didn't process that the most recent comments were from 2021 not from 2022. Sorry for resurrecting.