VsVim 2022 stack overflow in Vim.KeyNotationUtil #2919

mqudsi · 2021-07-25T22:20:22Z

Describe the bug
With VsVim for VS 2022 preview installed (under the preview 2 and preview 2.1), attempting to open any file in the solution explorer (by single-clicking or double-clicking) crashes VS 2022. WinDbg reports a stack overflow in VsVim.

To Reproduce
This reproduces for me every time with the default settings, because I didn't get a chance to configure anything. Unless it imports settings from the VS 2019 plugin? If so, let me know and I'll share those. Attempting to actually view the VsVim settings in VS2022 also triggers the crash.

Expected behavior
No crash.

Screenshots
N/A

Environment (please complete the following information):

Visual Studio version: 2022 preview 2.1
VsVim version: 2.10.0.0
Programming Language: C# files
Check(Type 'x') any that are installed:
- ReSharper
- Visual Assist

Additional context

This is the stack overflow per WinDbg when attached to VS 2022 preview with VsVim, when the crash triggers. It continues for miles and miles as the configured stack size is extremely large, but it's just repeating the same thing.

MethodDesc:   00007ff94a7d78d0
Method Name:  Vim.KeyNotationUtil+inner@259-11[[System.__Canon, mscorlib]].Invoke(Microsoft.FSharp.Collections.FSharpList`1<Vim.KeyInput>)
Class:        00007ff94a8209d0
MethodTable:  00007ff94a7d78f0
mdToken:      0000000006003a86
Module:       00007ff9481c2fe8
IsJitted:     yes
CodeAddr:     00007ff94a452d20
Transparency: Critical
MethodDesc:   00007ff94a7d78d0
Method Name:  Vim.KeyNotationUtil+inner@259-11[[System.__Canon, mscorlib]].Invoke(Microsoft.FSharp.Collections.FSharpList`1<Vim.KeyInput>)
Class:        00007ff94a8209d0
MethodTable:  00007ff94a7d78f0
mdToken:      0000000006003a86
Module:       00007ff9481c2fe8
IsJitted:     yes
CodeAddr:     00007ff94a452d20
Transparency: Critical
MethodDesc:   00007ff94a7d78d0
Method Name:  Vim.KeyNotationUtil+inner@259-11[[System.__Canon, mscorlib]].Invoke(Microsoft.FSharp.Collections.FSharpList`1<Vim.KeyInput>)
Class:        00007ff94a8209d0
MethodTable:  00007ff94a7d78f0
mdToken:      0000000006003a86
Module:       00007ff9481c2fe8
IsJitted:     yes
CodeAddr:     00007ff94a452d20
Transparency: Critical
MethodDesc:   00007ff94a7d78d0
Method Name:  Vim.KeyNotationUtil+inner@259-11[[System.__Canon, mscorlib]].Invoke(Microsoft.FSharp.Collections.FSharpList`1<Vim.KeyInput>)
Class:        00007ff94a8209d0
MethodTable:  00007ff94a7d78f0
mdToken:      0000000006003a86
Module:       00007ff9481c2fe8
IsJitted:     yes
CodeAddr:     00007ff94a452d20
Transparency: Critical
MethodDesc:   00007ff94a7d78d0
Method Name:  Vim.KeyNotationUtil+inner@259-11[[System.__Canon, mscorlib]].Invoke(Microsoft.FSharp.Collections.FSharpList`1<Vim.KeyInput>)
Class:        00007ff94a8209d0
MethodTable:  00007ff94a7d78f0
mdToken:      0000000006003a86
Module:       00007ff9481c2fe8
IsJitted:     yes
CodeAddr:     00007ff94a452d20
Transparency: Critical
MethodDesc:   00007ff94a7d78d0
Method Name:  Vim.KeyNotationUtil+inner@259-11[[System.__Canon, mscorlib]].Invoke(Microsoft.FSharp.Collections.FSharpList`1<Vim.KeyInput>)
Class:        00007ff94a8209d0
MethodTable:  00007ff94a7d78f0
mdToken:      0000000006003a86
Module:       00007ff9481c2fe8
IsJitted:     yes
CodeAddr:     00007ff94a452d20
Transparency: Critical
MethodDesc:   00007ff94a7d78d0
Method Name:  Vim.KeyNotationUtil+inner@259-11[[System.__Canon, mscorlib]].Invoke(Microsoft.FSharp.Collections.FSharpList`1<Vim.KeyInput>)
Class:        00007ff94a8209d0
MethodTable:  00007ff94a7d78f0
mdToken:      0000000006003a86
Module:       00007ff9481c2fe8
IsJitted:     yes
CodeAddr:     00007ff94a452d20
Transparency: Critical
MethodDesc:   00007ff94a7d62d0
Method Name:  Vim.KeyNotationUtil+inner@250-10T[[System.__Canon, mscorlib]].Invoke(Int32, Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Collections.FSharpList`1<Vim.KeyInput>,System.__Canon>)
Class:        00007ff94a81ff50
MethodTable:  00007ff94a7d62f0
mdToken:      0000000006003a8a
Module:       00007ff9481c2fe8
IsJitted:     yes
CodeAddr:     00007ff94a440000
Transparency: Critical
MethodDesc:   00007ff94a7d5878
Method Name:  Microsoft.FSharp.Core.FSharpFunc`2[[System.Int32, mscorlib],[System.__Canon, mscorlib]].InvokeFast[[System.__Canon, mscorlib]](Microsoft.FSharp.Core.FSharpFunc`2<Int32,Microsoft.FSharp.Core.FSharpFunc`2<System.__Canon,System.__Canon>>, Int32, System.__Canon)
Class:        00007ff94a754fb0
MethodTable:  00007ff94a7608d0
mdToken:      000000000600422a
Module:       00007ff9481c2fe8
IsJitted:     yes
CodeAddr:     00007ff94a43fae0
Transparency: Critical
MethodDesc:   00007ff94a7d4848
Method Name:  Vim.KeyNotationUtil.TryStringToKeyInputSet(System.String)
Class:        00007ff94a81e250
MethodTable:  00007ff94a7d48a8
mdToken:      0000000006003a74
Module:       00007ff9481c2fe8
IsJitted:     yes
CodeAddr:     00007ff94a43f680
Transparency: Critical
MethodDesc:   00007ff94a7d2480
Method Name:  Vim.Vim.LoadSessionDataCore[[System.__Canon, mscorlib]](System.__Canon)
Class:        00007ff94a737a88
MethodTable:  00007ff94a7409e8
mdToken:      000000000600271b
Module:       00007ff9481c2fe8
IsJitted:     yes
CodeAddr:     00007ff94a43dc70
Transparency: Critical
MethodDesc:   00007ff94a740530
Method Name:  Vim.Vim.LoadSessionData()
Class:        00007ff94a737a88
MethodTable:  00007ff94a7409e8
mdToken:      000000000600271c
Module:       00007ff9481c2fe8
IsJitted:     yes
CodeAddr:     00007ff94a43ced0
Transparency: Critical
MethodDesc:   00007ff94a7403b0
Method Name:  Vim.Vim.GetWindowSettingsForNewBuffer()
Class:        00007ff94a737a88
MethodTable:  00007ff94a7409e8
mdToken:      0000000006002706
Module:       00007ff9481c2fe8
IsJitted:     yes
CodeAddr:     00007ff94a423620
Transparency: Critical
MethodDesc:   00007ff94a740590
Method Name:  Vim.Vim.TryGetOrCreateVimBufferForHost(Microsoft.VisualStudio.Text.Editor.ITextView, Vim.IVimBuffer ByRef)
Class:        00007ff94a737a88
MethodTable:  00007ff94a7409e8
mdToken:      0000000006002722
Module:       00007ff9481c2fe8
IsJitted:     yes
CodeAddr:     00007ff94a41f960
Transparency: Critical
MethodDesc:   00007ff94a740968
Method Name:  Vim.Vim.Vim.IVim.TryGetOrCreateVimBufferForHost(Microsoft.VisualStudio.Text.Editor.ITextView, Vim.IVimBuffer ByRef)
Class:        00007ff94a737a88
MethodTable:  00007ff94a7409e8
mdToken:      000000000600274d
Module:       00007ff9481c2fe8
IsJitted:     yes
CodeAddr:     00007ff94a41f8d0
Transparency: Critical
MethodDesc:   00007ff94a7bdf18
Method Name:  Vim.UI.Wpf.Implementation.Mouse.VimMouseProcessorProvider.Microsoft.VisualStudio.Text.Editor.IMouseProcessorProvider.GetAssociatedProcessor(Microsoft.VisualStudio.Text.Editor.IWpfTextView)
Class:        00007ff94a7c1e18
MethodTable:  00007ff94a7bdf48
mdToken:      0000000006000145
Module:       00007ff94a2d0020
IsJitted:     yes
CodeAddr:     00007ff94a41f7e0
Transparency: Critical

The text was updated successfully, but these errors were encountered:

redcoreit · 2021-08-18T07:37:30Z

I'm experiencing similar problems. I've no dumps but I suppose it's the same issue.

Expected behavior
No crash.

Screenshots
N/A

Environment (please complete the following information):

Visual Studio version: 2022 preview 3.1 (same with 2.1)
VsVim version: 2.10.0.1
Programming Language: C# files
Check(Type 'x') any that are installed:
ReSharper (no)
Visual Assist (no)

jaredpar · 2021-08-22T17:22:41Z

Feel like this is probably specific to the type of project you are working on (else I should see it myself or it would be more broadly reported).

Can you tell me a bit more about what type of code you are working on here? Are you trying to open designer files, what language, etc ...? Anything that can help me lock down a repro.

mqudsi · 2021-08-23T02:35:15Z

It happens in both a ASP.NET Core project and a Windows desktop project. The only thing they have in common is that they're both C#.

I just remembered that I have a .vsvimrc file:

set vsvim_useeditordefaults
set shiftwidth=4
noremap gq :vsc Edit.FormatSelection<CR>
nnoremap <c-s> :w<CR>
inoremap <C-s> <Esc>:w<CR>
nnoremap gcc :vsc Edit.ToggleLineComment<CR>
vnoremap gc :vsc Edit.ToggleComment<CR>
" reformat on paste
nnoremap p pv`.=
" reformat on pasting over selected text
" vnoremap p "_dPv`.=

jaredpar · 2021-08-23T04:05:46Z

Can you share out %localappdata%\VsVim\vimdata.json ?

Looking at the stack it's crashing parsing old session data. There is a macro in there that is causing VsVim to hit a strange parse error.

After sharing if you delete that file then you should be able to use VsVim just fine.

mqudsi · 2021-08-23T19:39:33Z

That worked. I didn't realize that vsvim persisted macros past the session. Here's mine, it has some ugly stuff (but nothing that makes vsvim for VS2019 crash):

vimdata.zip

Kethku · 2021-08-25T23:21:34Z

Just FYI, this fixed the issue I was running into when I first tried vsvim as well. Thanks for the tip

jaredpar · 2021-08-27T00:07:49Z

I didn't realize that vsvim persisted macros past the session.

I honestly forgot myself until I was poking around in the call stack here.

Just FYI, this fixed the issue I was running into when I first tried vsvim as well. Thanks for the tip

Pretty sure what happened here is that I uploaded the Debug bits of VsVim for this release. Mostly cause there was a lot of code churn hence wanted users to file bugs if they hit any Debug.Assert failures. This particular bit of F# code is relying on the recursion to be re-written to iteration. That may not happen in Debug builds (can't remember off hand). That would explain why it is all the sudden happening here as I didn't touch that part of the code for VS 2022.

redcoreit · 2021-08-29T09:42:16Z

Can you share out %localappdata%\VsVim\vimdata.json ?

Looking at the stack it's crashing parsing old session data. There is a macro in there that is causing VsVim to hit a strange parse error.

After sharing if you delete that file then you should be able to use VsVim just fine.

Yes, this solved my issue as well, thank you.

nernst · 2021-09-11T17:13:20Z

Just FYI, this fixed the issue I was running into when I first tried vsvim as well. Thanks for the tip

Same for me. FWIW, I was opening a brand new C# Class Library that only had a single, default class in it. Nothing special with the project. Removing the vimdata.json unblocked me.

mqudsi · 2021-09-11T20:17:12Z

@jaredpar you know your code best; do you think it’s likely that such a high recursive invocation count was reached without there being an underlying issue and that it correctly results in such a high-O so as to overflow the (huge) stack size, but not an infinite one (w/ the tail-call optimization)?

adamency · 2021-12-01T13:45:20Z

Confirming issue happening when upgrading from VS 2019 to 2022 which provide incompatible versions of VsVim.

Also confirming issue is solved by deleting the file in %localappdata%/VsVim/vimdata.json (just rename it if you want to backup your VsVim data for VS 2019).

mqudsi changed the title ~~VsVim 2022 stack overflow in F# key binding settings~~ VsVim 2022 stack overflow in Vim.KeyNotationUtil Jul 25, 2021

jaredpar added the need more data label Aug 22, 2021

redcoreit mentioned this issue Sep 17, 2021

Visual Studio 2022 support msomeone/PeasyMotion#24

Closed

nosami mentioned this issue Nov 30, 2021

VsVim2022 is crashing VS 2022 #2942

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

VsVim 2022 stack overflow in Vim.KeyNotationUtil #2919

VsVim 2022 stack overflow in Vim.KeyNotationUtil #2919

mqudsi commented Jul 25, 2021

redcoreit commented Aug 18, 2021 •

edited

Loading

jaredpar commented Aug 22, 2021

mqudsi commented Aug 23, 2021

jaredpar commented Aug 23, 2021

mqudsi commented Aug 23, 2021 •

edited

Loading

Kethku commented Aug 25, 2021

jaredpar commented Aug 27, 2021

redcoreit commented Aug 29, 2021

nernst commented Sep 11, 2021

mqudsi commented Sep 11, 2021 •

edited

Loading

adamency commented Dec 1, 2021

VsVim 2022 stack overflow in Vim.KeyNotationUtil #2919

VsVim 2022 stack overflow in Vim.KeyNotationUtil #2919

Comments

mqudsi commented Jul 25, 2021

redcoreit commented Aug 18, 2021 • edited Loading

jaredpar commented Aug 22, 2021

mqudsi commented Aug 23, 2021

jaredpar commented Aug 23, 2021

mqudsi commented Aug 23, 2021 • edited Loading

Kethku commented Aug 25, 2021

jaredpar commented Aug 27, 2021

redcoreit commented Aug 29, 2021

nernst commented Sep 11, 2021

mqudsi commented Sep 11, 2021 • edited Loading

adamency commented Dec 1, 2021

redcoreit commented Aug 18, 2021 •

edited

Loading

mqudsi commented Aug 23, 2021 •

edited

Loading

mqudsi commented Sep 11, 2021 •

edited

Loading