When I saw the video, I immediately thought that this must be a recent regression. But in my attempt to track down the first bad commit, I have tried with our releases from the past 6 months, and then jumped all the way back to 1.40.2 from October 2019 and I always get the current behavior.

Basically, changing the text buffer never ran a viewport tokenization, which is quite an oversight, but not a recent regression.

@alexdima You're right, if start VS Code on a large file (like checker.ts) and edit towards the end of the file, colorization doesn't happen for the first few seconds or so. But with 1.63 it takes more like a minute before the white text gets colorized. I just verified by reverting to 1.61. There's definitely a big change in behavior.

@ahejlsberg You're right, tokenization is now running for 2ms, and then yielding for 14ms, making it roughly 8 times slower:

I think this bad yielding behavior is caused by #137646 , where we moved away from setImmediate and to requestIdleCallback.

fyi @jrieken @bpasero

I've created #139072 to track that typing does not trigger viewport tokenization and I suggest to use this issue to track that background tokenization takes a lot longer now.

Steps to verify:

• open checker.ts
• have minimap enabled so you can see when background tokenization reaches the bottom
• scroll to the bottom and then do a window reload
• tokenization should reach the bottom in ~10-15s
• (you can optionally take a JS CPU trace and check that there aren't a lot of idle gaps in there)

Verified. Took ~11 seconds on my machine.