TS Server performance 10-50mb files

[mjbvz]

From #63473

TypeScript Version: 3.3.0-dev.20181128

Search Terms:

• TS Server

Repo
Open one of the files from the user's project. These are very large files, some with 1.5 million lines

Bug
TS Server goes unresponsive. Here's one example log file:
tsserver.log

Two observations:

• We generate a massive outlining spans response
• Then this error happens:

Err 306   [18:52:6.115] Exception on executing command {"seq":12,"type":"request","command":"navtree","arguments":{"file":"/Users/matb/Downloads/www/code/80000.chunk.js"}}:

    No Project.

    Error: No Project.
        at Object.ThrowNoProject (/Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:120007:23)
        at IOSession.Session.getFileAndLanguageServiceForSyntacticOperation (/Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:125190:42)
        at IOSession.Session.getNavigationTree (/Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:125539:31)
        at Session.handlers.ts.createMapFromTemplate._a.(anonymous function) (/Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:124455:61)
        at /Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:125847:88
        at IOSession.Session.executeWithRequestId (/Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:125838:28)
        at IOSession.Session.executeCommand (/Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:125847:33)
        at IOSession.Session.onMessage (/Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:125869:35)
        at Interface.<anonymous> (/Users/matb/Downloads/www/node_modules/typescript/lib/tsserver.js:127129:27)
        at emitOne (events.js:116:13)
        at Interface.emit (events.js:211:7)
        at Interface._onLine (readline.js:282:10)
        at Interface._normalWrite (readline.js:424:12)
        at Socket.ondata (readline.js:141:10)
        at emitOne (events.js:116:13)
        at Socket.emit (events.js:211:7)
        at addChunk (_stream_readable.js:263:12)
        at readableAddChunk (_stream_readable.js:250:11)
        at Socket.Readable.push (_stream_readable.js:208:10)
        at Pipe.onread (net.js:594:20)

[AnyhowStep]

1.5 million!
I'm super curious about what's in those millions of lines

[mjbvz]

I'm investigating why we start making requests on the 80000.chunk.js file even though it is never opened on the TS Server

(Note that the stall seems to happen before this and on a different file)

[mjbvz]

The originally file was 50.2 mb. Seems like a VS code error that caused us to start making requests for this file without opening it first. 50mb is the limit of what we synchronize with extensions. I haven't been able to trigger it again though

The smaller, 10-15mb files in the repo are likely more interesting. Here's the log:
tsserver.log.zip

It's seems faster with 3.3 and I don't see any stalls. The long running requests:

• Initial open: 8486.1065ms
• navtree: 6603.4369ms
• completionInfo: 1463ms

[sheetalkamat]

@mjbvz On our side the large file is classified as anything more than 4mb. https://github.com/Microsoft/TypeScript/blob/master/src/server/editorServices.ts#L4

[mjbvz]

Does that mean we should not sync files over that size with TS Server?

[sheetalkamat]

Does that mean we should not sync files over that size with TS Server?

Right.

[mjbvz]

We can do that. On the server side, should requests against large files be treated as noops? Right now it looks like ts server still tries handling them

[RyanCavanaugh]

@mjbvz what's left to do here?

[mjbvz]

I'll test this again in the latest VS Code