Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Editor lags badly when Vim extension is enabled #1800

Closed
Maistho opened this issue Jun 15, 2019 · 47 comments
Closed

Editor lags badly when Vim extension is enabled #1800

Maistho opened this issue Jun 15, 2019 · 47 comments

Comments

@Maistho
Copy link

Maistho commented Jun 15, 2019

Hi, I notice severe lag/freezing when typing in dart files in my flutter project. The lag is not there if I disable the Dart vscode extension.

Sometimes inputs are lost, most often when pressing escape (I use the vim plugin) and then quickly do some vim commands. I assume it is because the CPU is busy, since the extension host seems pinned at 25% (on a quadcore system).

I use an overclocked i5-6600k and 32GB of RAM, so my system specs should be more than enough.

I did some typing in a file and recorded the CPU profile which is attached below.

Any ideas what causes this?
CPU-20190615T122227.237Z.cpuprofile.txt
CPU-20190615T121759.944Z.cpuprofile.txt

@DanTup
Copy link
Member

DanTup commented Jun 15, 2019

The CPU profiles don't look too bad, but it's not clear if these are just normal typing, or if you saw the issue during the capture of these.

A few other questions:

Can you also confirm what version of VS Code, the Dart extension and Dart/Flutter you're using?

Is it always Vim keystrokes that get dropped, or typing in the editor too? (in theory the extension host shouldn't be able to prevent typing in the editor, but a busy extension host could prevent - or at least delay - Vim handling commands).

If you can reliably repro it, could you try turning off autoImportCompletions and see if it makes any difference (my guess is that it will, as I suspect the increased CPU is from the bigger completion list).

@DanTup DanTup added the awaiting info Requires more information from the customer to progress label Jun 15, 2019
@kika
Copy link

kika commented Jun 15, 2019

I started to have the same problem recently. I can't say who to blame because I auto update the extension and frequently update flutter. I'm on the dev channel and always at the latest DartCode release. The issue becomes annoying, I frequently type blind for 10-15 characters while the editor lags behind and this behavior interferes with autocomplete. Like I'm typing if(something.something == null), the editor lags behind while trying to figure out null from nu and then I end up with if(something.something == nu)ll, lol. I also use vim plugin and I'm on a MacPro with 3.6GHz 8 cores (16 threads) with 64GB RAM. If this is not enough for the text editor, I don't know what is :-)
For me it's both vim commands and the regular typing in insert mode, but dropping altogether happens rather rarely, it's the huge lag that frustrates me the most. My project is 10K Dart LOC in 77 files.

P.S. Disabling autoImportCompletions did help!.

@DanTup
Copy link
Member

DanTup commented Jun 16, 2019

the editor lags behind

Do you mean your keystrokes aren't even appearing as characters on the screen immediately as you type? It was my understanding that extensions couldn't affect this at all (since they run in another extension and everything is async), so that's quite concerning.

Or do you mean that other things (like vim commands, code completion, red squiggles) are delayed?

A CPU profile capture while this lagging is going on would be really helpful (maybe of both Vim and Dart extensions - I'm not sure if one profile includes both).

Also, if you can think it's easy to repro for a non-Vim user by installing the extension and following some steps, let me know and I can have a go.

Also - please confirm your VS Code + Extension versions. There was a serious memory leak in the previous VS Code, but that should be fixed in v1.35.

@dat7e
Copy link

dat7e commented Jun 16, 2019

I also have this problem. For me, CPU spikes happen every time the Dart plugin tries to display autocomplete suggestion. So for example when I type "Con" for "Container", the editor is lagged at "C" (my keystrokes won't be shown on screen until the suggestion box appears).

I'm on the latest version of VSCode, Dart plugin, Flutter, and Vim plugin. Computer is a maxed-out Macbook Pro 15 2015

Additional notes:

  • The problem persists even when vim is disabled with: "vim.disableExtension": true
  • Turning off autoImportCompletions also doesn't seem to have any effect.

@Maistho
Copy link
Author

Maistho commented Jun 16, 2019

@DanTup

The CPU profiles don't look too bad, but it's not clear if these are just normal typing, or if you saw the issue during the capture of these.

I saw the issue during the capture of these, while typing.

Can you also confirm what version of VS Code, the Dart extension and Dart/Flutter you're using?

VSCode 1.35.1
Dart-Code 3.1.0
Flutter 1.7.3
Dart 2.3.2-dev.0.1.flutter-6e0d978505

Is it always Vim keystrokes that get dropped, or typing in the editor too?

Hm, I think that only Vim keystrokes get dropped now that you say it. I can't recall it happening with regular typing. Regular typing still lags behind a lot, but I don't think any inputs are dropped.

could you try turning off autoImportCompletions and see if it makes any difference

Doesn't make a difference for me

Edit: after restarting vscode it seems to make a difference, typing seems much more responsive wih it turned off. I added a screencast showing this example as well.

Do you mean your keystrokes aren't even appearing as characters on the screen immediately as you type?

Exactly. I'll attach a screencast showing the issue.

With the extension active: https://youtu.be/oMb3U2sFBGs

With the extension active and autoImportCompletions turned off: https://youtu.be/vBYSwlvjgQU

Without the flutter/dart extensions active: https://youtu.be/DuB5oXc5TRQ

I'm typing via a macro on my keyboard, which waits 100ms between each keystroke. They should be evenly spaced as in the example where the extensions are disabled.

@kika
Copy link

kika commented Jun 16, 2019

the editor lags behind
Do you mean your keystrokes aren't even appearing as characters on the screen immediately

Correct. I also thought that VS Code should pass the characters through immediately, but this is not the case.

Or do you mean that other things (like vim commands, code completion, red squiggles) are delayed?

This too. This is obviously somehow related to completion.

A CPU profile capture while this lagging is going on would be really helpful (maybe of both Vim and Dart extensions - I'm not sure if one profile includes both).

I'll try to make it.

Also, if you can think it's easy to repro for a non-Vim user by installing the extension and following some steps, let me know and I can have a go.

I can't repro it on a small project with or without vim. Thus I thought it might be related to some caches, cruft, artifacts, etc, tried to clean as much as possible, but to no avail.

Disabling autoImportCompletions makes my life much better, but doesn't completely alleviate the problem. I still type faster than VS Code can process my keystrokes and I'm not a fast typist.

Also - please confirm your VS Code + Extension versions. There was a serious memory leak in the previous VS Code, but that should be fixed in v1.35.

Everything latest. VS Code 1.35.1, DartCode 3.1.0

@DanTup
Copy link
Member

DanTup commented Jun 17, 2019

Thanks for all the info. I can repro this with the Vim extension installed, and it seems like it's something that Vim is doing. For example, you can repro this by just creating a dummy VS Code extension that blocks for 1sec when asking for completions:

context.subscriptions.push(vscode.languages.registerCompletionItemProvider({ scheme: "file" }, {
	provideCompletionItems(document: vscode.TextDocument, position: vscode.Position, token: vscode.CancellationToken, context: vscode.CompletionContext): vscode.ProviderResult<vscode.CompletionItem[] | vscode.CompletionList> {
		var start = Date.now();
		let i = 0;
		while (Date.now() < start + 1000) {
			// Block for a second...
			i++;
		}
		return [new vscode.CompletionItem(`aaaItem (${i} iterations)`)];
	},
}));

Without the Vim extension, this does not affect typing at all - the completion request runs in the extension host, and the UI is completely responsive. However if you enable the Vim extension, this causes the editor to lag. I don't exactly what's going on, but I keep seeing running save participants in the status bar, so I wonder if it's triggering something that requires VS Code to block the editor, but because the extension host is busy, it's blocked for that whole duration.

Blocking the extension for long periods is not ideal, however since there's only one extension host and it's single threaded, this does happen when generating huge completion lists (not just because of our code, but also the JSON serialisation VS Code does).

I'll see if I can figure out exactly what's happening to trigger this, to see if it can be avoided.

In the meantime, it's worth trying out this new build:

https://github.com/Dart-Code/Dart-Code/releases/tag/v3.2.0-alpha.1

It was some performance tweaks relating to completion, so it might make the problem a little less severe (as will turning off auto-import-completions).

@DanTup DanTup added is performance other extension Something caused by another extension and removed awaiting info Requires more information from the customer to progress labels Jun 17, 2019
@DanTup DanTup changed the title Extension using up a lot of CPU time when typing Editor lags badly when Vim extension is enabled Jun 17, 2019
@DanTup
Copy link
Member

DanTup commented Jun 17, 2019

Ok, I think I tracked this down. VS Code Vim is overriding the VS Code command that handles typing:

https://github.com/VSCodeVim/Vim/blob/1ef304e5dd857c93894c117990e2b58cb0a6abbc/extension.ts#L293

Based in this comment that's an ok thing to do, it was specifically added to support Vim. However, it means that every key press has to go via the extension host, and that means if any extension blocks for any period of time, it will lag the typing. In the case where you press a character that triggers completion, that means the character will lag until the completion request has been satisfied.

I don't think there's really anything we can do to fix this (besides some of the tweaks in the Alpha version above, which will yield with await periodically during processing). We can (and do) try to make completion as fast as possible, but that typing is being handled in the extension host feels like a really bad design and I think the real fix would be that extensions that need to handle typing should get their own thread (extension host), since extensions often need to do intensive processing (like building and serialising tens of thousands of objects).

I've opened an issue with VS Code to see if they're aware of these sorts of issues and have had any thoughts:

microsoft/vscode#75627

I'll keep this issue open for a little while in case they have simple ideas that might improve this.

@kika
Copy link

kika commented Jun 19, 2019

I've installed the alpha and reenabled autoImportCompletions. The result is almost perfect. It's still a little sluggish, but bearable. Thank you!

@DanTup
Copy link
Member

DanTup commented Jun 19, 2019

@kika glad to hear! Most likely the improvements come from adding some awaits in the loop processing the completions - this was mainly done so we can check if the request is cancelled and save some work, but it likely also has the effect that we don't block the thread for such long periods and Vim is able to process the keys more frequently.

We might be able to tweak this further (add more awaits if there is particular code that blocks the CPU for a while), so grabbing CPU profiles (ideally that just cover an example of typing where this was observed) for your specific case(s) is useful.

  • Run the Show Running Extensions command
  • Right click Dart and choose Start Extension Host Profile
  • Reproduce the issue
  • Back in extension list, Right click Dart and choose Stop Extension Host Profile
  • Right click Dart and click Save Extension Host Profile

@kika
Copy link

kika commented Jun 30, 2019

@DanTup I waited until it lands in the release, installed 3.2.0 and noticed sluggishness again. Maybe I just got used to how it's better now, readjusted and it's not worse than in alpha, I just demand better :-)
Anyway, here's the profile.
CPU-20190630T231203.767Z.cpuprofile.txt

@DanTup
Copy link
Member

DanTup commented Jul 8, 2019

@kika thanks for the profile. There are a couple of long pauses in there, some around 300ms. It's in the construction of the DelayedCompletionItems from the included suggestion sets.

I think there are two things that might make this better:

  1. Caching the CompletionItems we create (something I started on a branch https://github.com/Dart-Code/Dart-Code/compare/completion-performance-improvements?expand=1#diff-dbdb8d4706e429026a91e064970b05b7 but it didn't have measurable differences in my testing).
  2. Yielding the thread during the mapping (we already do this in some loops, but there's one where we don't as the code isn't currently async).

How reliably can you reproduce this? If I made a custom build of the extension for you to test, would you be able to easily tell whether there's a significant improvement?

@kika
Copy link

kika commented Jul 8, 2019

@DanTup It happens pretty regularly, so I think I can reproduce this. I'm willing to test the build, thank you.

@DanTup
Copy link
Member

DanTup commented Jul 10, 2019

@kika there's a build here:

https://github.com/Dart-Code/Dart-Code/releases/tag/v3.3.0-alpha.2.with-completion-awaits

Download the .vsix, then run the Extensions: Install from VSIX command in VS Code and browse to that file (then click reload when prompted). It will yield with an await after every 100 items within each suggestion set. In my testing, it roughly halved the maximum time the thread was blocked (from around 70ms to around 32ms).

If that makes it better, but still not good enough, then I might consider adding an option that lets you set how frequently it'll await and you can play around with values (probably I'd ship it with a conservative default, like 100-500 items, but you could lower it for better responsiveness at the expense or slightly increasing the total time to build the list).

@kika
Copy link

kika commented Jul 13, 2019

@DanTup After a couple of days of testing I can't say this version is any better :-( I feel it as even worse, but it forced me to upgrade to 1.36, so it could be something else.
I was just able to type this.conv, before I've seen the first character (space, the cursor moved).

@kika
Copy link

kika commented Jul 14, 2019

Reinstalled 3.2.0 and it's indeed better, less sluggish.

@DanTup
Copy link
Member

DanTup commented Jul 15, 2019

Hmmm, I can't understand why that would be (and for me it did seem better). If you have time, would you be able to grab profile traces from each doing roughly the same thing so I can compare them directly? It's possible I misinterpreted where the delays were.

@kika
Copy link

kika commented Aug 15, 2019

@DanTup Sorry was away on vacations. Got back, installed all vscode updates and updated to 3.3. No problems so far!

@DanTup
Copy link
Member

DanTup commented Aug 19, 2019

I didn't include the changes above in v3.3 since it sounded like they made things worse. I'm still not sure why though - so if you continue to see things be bad, let me know and I can rebase those changes and add some more logging so we can dig deeper.

@Timmmm
Copy link

Timmmm commented Nov 20, 2019

I have this issue but don't use the Vim extension. I eventually narrowed it down to the rust analyzer extension (I have a Rust backend in my project).

That also overrides the typing handler. Fortunately it's behind an option so this fixes it (and as a bonus disables "enhanced typing" which is really annoying).

"rust-analyzer.enableEnhancedTyping": false,

@DanTup
Copy link
Member

DanTup commented Nov 21, 2019

Thanks for the note! I hadn't considered that other languages extensions could also cause this if they happen to be active. FWIW though, the Rust activation events look sane:

https://github.com/rust-analyzer/rust-analyzer/blob/612a72fc4ea4376920f2a7da7b3c334227c1716c/editors/code/package.json#L57

So in theory it shouldn't be active when you're working on Dart projects as long as you don't have Rust files also in the same project?

@Timmmm
Copy link

Timmmm commented Nov 22, 2019

as long as you don't have Rust files also in the same project?

I do! Doesn't look like separating them out into separate projects would help much either because it also activates if you have a Rust project in the same workspace (unless this is a really badly named variable):

  "workspaceContains:**/Cargo.toml"

Especially annoying that they use ** because it actually only looks in the root directory for Cargo.toml and is constantly nagging me that it can't find it!

@DanTup
Copy link
Member

DanTup commented Nov 24, 2019

because it also activates if you have a Rust project in the same workspace

That's right - extensions are per-window in VS code and not per-workspace-folder, so having a Rust project open at all will activate the extension, and that will result in every keypress in an editor going through the extension host. For something like Vim this makes sense, but if languages are using this hook for language handling, then it's not so good (it'd be better if they could scope it per-language).

Especially annoying that they use ** because it actually only looks in the root directory for Cargo.toml and is constantly nagging me that it can't find it!

Yeah, that is a bit weird to activate with it anywhere, but then require it at the top. To be fair though, handling multi-project workspaces is a bit tricky - VS Code has multi-root workspaces, but they don't seem to be very well thought out for having projects in sub folders. Previously I recommended multi-root workspaces for Dart, but because of microsoft/vscode#45470 I don't really anymore - instead we try to make the extension find projects at any level and let the user select between them where required.

Hopefully VS Code will get a better design for this in future (or other extensions will stop using it by default for non-critical functionality)!

@DanTup
Copy link
Member

DanTup commented Mar 23, 2020

I tested the code above in VS Code/nodejs too (this is where this code is running for real in the extension), and it behaved the same:

Screenshot 2020-03-23 at 15 04 56

I'm not sure why you'd see different behaviour in Flutter release mode (it may be worth raising in case it's considered a bug).

Adding setTimeout(0) would likely slow things down for the general case (non-Vim users) though, because based on my testing above, setTimeout(0) may introduce a real delay (beyond just waiting for the event loop to continue) and we're doing it a lot of times (every 100 completion items).

@Kavantix
Copy link

Ah nevermind this is not dart ofcourse ;)

@sdykae
Copy link

sdykae commented May 30, 2020

any updates ;(?
For some reason neovim extension and simplevim don't have this problem.

@kika
Copy link

kika commented May 30, 2020

@sdyalor Thanks!
neovim is not affected because as they say:
First-class VSCode insert mode. The plugin unbinds self from the type event in the insert mode, so no typing lag and freezing anymore when long completion popup appears.
Neovim runs a full-blown real neovim behind the scenes, so it's not emulation. It's like a frontend to real neovim. I'm going to try it.

@minvs1
Copy link

minvs1 commented May 30, 2020

It seems that 3.11 re-introduced this issue, going back to 3.10.2 removes the lag.

May-31-2020 02-32-43

CPU-20200530T232815.618Z.cpuprofile.txt

@DanTup
Copy link
Member

DanTup commented Jun 1, 2020

It seems that 3.11 re-introduced this issue, going back to 3.10.2 removes the lag.

This issue hasn't ever really been fixed, I don't currently have any more ideas to improve things without some input from VS Code. Could this just be timing? If you're typing during initial analysis, the extension host will be quite busy and more likely to lag. If you wait until the initial analysis (and preloading of completions, which isn't always obvious because it doesn't trigger the "Analyzing..." status notification) completes, it's likely to be much better.

In the profile, it's hard to tell where the time is (since the code is webpacked), but I think it could be caused by the completion preload (it's running a bunch of callbacks).

If you switch back to 3.11, open a project, then wait for 60 seconds after the "Analyzing..." notification is gone, do you still see it lag worse than before?

@DanTup DanTup added the blocked on vs code / lsp / dap Requires a change in VS Code to progress label Jun 1, 2020
@minvs1
Copy link

minvs1 commented Jun 2, 2020

If you switch back to 3.11, open a project, then wait for 60 seconds after the "Analyzing..." notification is gone, do you still see it lag worse than before?

Even waiting for more than 10 mins lag still persists

@DanTup
Copy link
Member

DanTup commented Jun 4, 2020

I reviewed the diff between v3.10 and v3.11 but I've only identified one change that might impact completion performance (#2468), but in my testing it doesn't to change the timings (it generally takes 60ms-180ms to build the completion list in either case).

Can you give exact steps how I might repro this? I've installed the Vim extensions, and v3.11 of the Dart extension, I opened a project and started typing, but I see no latency whatsoever and I'm not sure if that means I'm not in the right mode or something?

@DanTup
Copy link
Member

DanTup commented Jun 4, 2020

Scratch that - I had a Dart project open not Flutter, so that had massively reduced the number of symbols.

I tested with both v3.10 and v3.11 with some logs to write timings in the completion provider, but I saw the same timings (and same stuttering) in both and v3.11 didn't seem any worse (it's not good, but it didn't feel like it'd gotten worse).

Would be able to capture a profile trace from both versions (performing the same actions in the same project) so I can compare them to see if the time is going somewhere other than the completion handler?

https://github.com/microsoft/vscode/wiki/Performance-Issues#profile-the-running-extensions

@minvs1
Copy link

minvs1 commented Jun 4, 2020

Actually, when I started doing those profiles I couldn't replicate version lag difference, lag being about the same. The only difference is when memory usage is about 60% (because of the browser, slacks, etc) lag becomes much worse (sorry for confusing you with versions there).

Attaching profiles with and without high memory usage.

High memory usage:
3.11 - CPU-20200604T203517.142Z.cpuprofile.txt
3.10.2 - CPU-20200604T204009.066Z.cpuprofile.txt

Low memory usage:
3.11 - CPU-20200604T194229.801Z.cpuprofile.txt
3.10.2 - CPU-20200604T194446.791Z.cpuprofile.txt

@DanTup
Copy link
Member

DanTup commented Jun 8, 2020

I compared the profiles and there wasn't a huge difference, but there are some additional (and longer) Garbage Collector pauses when there's low memory (which I guess makes sense):

Screenshot 2020-06-08 at 11 08 30

There were some additional events (like updating codelens) that appeared in the high memory one (also adding additional time) that weren't always in the low memory one (though that could just be a timing or something).

Soon we'll be moving to LSP, which it just occurred to me will move some of this processing out-of-process (and therefore consume less time in the extension host thread). Could you try enabling dart.previewLsp and see what difference this makes? I also have some potential upcoming changes that may significantly improve that too (removing 90% of the number of completion items) - that will also be LSP-only.

@minvs1
Copy link

minvs1 commented Jun 11, 2020

I tried on two separate machines (but synced settings) which had the same issue. Turning LSP seems to fix this on both of them (still has some minor lag but completely doable). One machine, which profiles I am attaching below, had plenty of free resources, but the lag was still present without LSP so, maybe low on the memory (or other resources) wasn't the issue?

Without LSP - CPU-20200611T123908.696Z.cpuprofile.txt

With LSP - CPU-20200611T124136.775Z.cpuprofile.txt

Thanks for the help!

@DanTup
Copy link
Member

DanTup commented Jun 17, 2020

Turning LSP seems to fix this on both of them

Glad to hear! The goal is to move over to LSP by default in an upcoming release, so it sounds like that will help some. I've also recently landed a change in LSP (which hasn't made it to Flutter yet, but I hope will be in the master branch soon) that should reduce the number of completions by filtering based on the prefix (so when you press p we'll only transmit completions that start with p rather than every completion - we did this previously because the spec is vague about how backspacing is handled). On average I think this should result in 1/26th of the number of completions (which hopefully means the stutter is 1/26th as long).

I'll ping here once that change is in Flutter's master branch so you can try it out.

@willhaslett
Copy link

willhaslett commented Jun 22, 2020

fwiw, I too enabled dart.previewLsp, restarted, and things are much better. I was in barely-usable territory. It's still a bit laggy, but way better.

Update: sort of. It seems to particularly struggle when typing arguments.

@DanTup
Copy link
Member

DanTup commented Jun 23, 2020

@willhaslett which Flutter branch are you on? A few of my changes made it to Flutter's master branch yesterday, and I've landed another change in the SDK today (which might take a few days/weeks to reach Flutter master).

If you're still seeing stutters on master, can you give a specific code example where you see this so I can take a look? Thanks!

@willhaslett
Copy link

@DanTup I'm on master. Just upgraded. Will report back.

@willhaslett
Copy link

@DanTup way better after upgrade! Fixed for me.

@DanTup DanTup added this to the v3.13.0 milestone Jun 29, 2020
@DanTup DanTup removed blocked on vs code / lsp / dap Requires a change in VS Code to progress other extension Something caused by another extension labels Jun 29, 2020
@DanTup DanTup modified the milestones: v3.13.0, v3.14.0 Jul 28, 2020
@DanTup
Copy link
Member

DanTup commented Aug 24, 2020

I'm going to consider the fix for this to be using LSP mode. It's still behind a preview flag for now, but I hope it can be enabled by default in the coming months (likely after the next Flutter stable release, so the fixes mentioned above are available to stable users) and the non-LSP code path won't be developed after that (and ultimately will be removed).

If anyone still sees editor/typing lag when using LSP and a recent build (dev/master for Flutter), please do let me know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

9 participants