tsc on tsconfig/all exceeds the default memory limit

[samreid]

Similar to #1415. Today @zepumph @jonathanolson and CT all reported heap memory errors while running tsc on tsconfig/all. For instance, CTQ reported:

error testing tsc: 
<--- Last few GCs --->

[1152858:0x59d1230]   205600 ms: Mark-sweep 4041.9 (4136.3) -> 4026.5 (4136.8) MB, 1519.5 / 0.0 ms  (average mu = 0.173, current mu = 0.049) allocation failure; scavenge might not succeed
[1152858:0x59d1230]   207236 ms: Mark-sweep 4042.5 (4136.8) -> 4026.4 (4136.8) MB, 1596.5 / 0.0 ms  (average mu = 0.104, current mu = 0.024) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb7a940 node::Abort() [node]
 2: 0xa8e823  [node]
 3: 0xd5c940 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xd5cce7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xf3a3e5  [node]
 6: 0xf3b2e8 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [node]
 7: 0xf4b7f3  [node]
 8: 0xf4c668 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 9: 0xf26fce v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
10: 0xf28397 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
11: 0xf0956a v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
12: 0x12ce7af v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
13: 0x16fb6b9  [node]

@zepumph and I knew the memory overhead was near for #1415 but we were surprised to see it already happening in tsc. Though we acknowledge there is variability from run to run (even though there is no caching or incrementaling).

In testing with @jonathanolson we saw that we might buy some time by running tsc in a separate script. Having lint and other things running in the same script may lower the memory ceiling a bit. This worked for a script that @jonathanolson was using, but CTQ already runs it in an isolated script.

A short term workaround is to boost the Node heap space, but this is tricky since it must be done on every developer machine (including open source developers), every server and service, such as bayes and the production servers, etc. We do not currently have one place where we factor out a call to tsc so we don't have a place where we can uniformly boost that memory. Different developers do this different ways at the moment.

@zepumph and I believe the best solution to this will be to move to incremental type checking which is described in #1424 which is blocked by phetsims/tasks#1132.

Assigning to @marlitas for consideration in scheduling.

[zepumph]

Yesterday during dev meeting, I kinda looped this issue into #1424 in our priorities discussion. This is a big reason to increase the priority of #1424, though not the only reason to work on that issue (phet-lib/doc/speedup/etc).

[marlitas]

I thought this was handled? However I really am not attuned to all the details of this issue. Bringing back to dev meeting to discuss.

[marlitas]

Discussed at dev meeting. We feel this can be closed since this error is now gone. Remaining work in: #1356