"dotnet paket --version" hangs until forcefully closed

[JohnyWS]

Description

Short version

Some threads hang at dotnet paket --version and never moves on. This results in builds hanging because some projects never gets build; Visual Studio won't unload projects or ReSharper fails to load; possibly also causing other faulty behaviors.

Longer version

We've found that having the CPU max out sometimes causes dotnet paket --version commands to hang a process. It seems to be some kind of timing issue, but we are uncertain if the issue is in dotnet faulting on the output from paket, og if paket simply never exists correctly.

The problem seems to only happen when using the new SDK project format (which is the one leveraging dotnet to begin with), as we do not have the same issue with our large Full Framework projects that have not yet been upgraded.

Also, having anti virus monitoring file changes, as well as using ReSharper, seems to increase the likelyhood dramatically of it happening.

We started noticing dotnet processes not closing down again after calling dotnet paket --version, and they seem to be what is blocking all the processes. Even more odd is, that when inspecting the process with Process Explorer and going to the Threads tab, the process finally closes down. What exactly causes the process to close down from that inspection we have still not uncovered, as listing the threads of the process via PowerShell does not yield the same result.

Repro steps

1. Have a large solution (in our case 230 projects) in Full Framework, but in the new SDK format.
2. Have ReSharper active in Visual Studio.
3. Work for a bit in Visual Studio, building and navigating around for 4-5 minutes.
4. Attempt to unload the solution

Expected behavior

Visual Studio should simply unload the solution.

Actual behavior

Unloads hangs at "Unloading project 0 of 230 ..." and never quits (have tried leaving it for multiple hours). In Process Explorer a process will be running with the command dotnet paket --version.

Known workarounds

Either:

1. Shut down the hanging process by going to the threads tab, or forcing the process to exit.
   Or:
2. Removing the part in the paket.targets file that actually calls dotnet paket --version, and replacing it with calling the local paket.exe always instead. This get overridden however when paket install/update is called, and must be corrected again, and there are no switches to prevent it from happening.

Final thoughts

This issue has been downright cripling for our development department, as everything slowed down to an almost standstill. Different developers saw the bug more or less frequently, but some had an unresponsive Visual Studio for 10-15 minutes at a time.

We do not know, however, if the error is with dotnet instead of directly with paket, but it is only in combination of the two we have the error.

The best solution, for us right now, I believe is to have some sort of flag, that would brevet paket to get asked all the time.

I hope the issue makes somewhat sense, and that a more permanent solution can be found, rather than we manually fix the targets file, time and time again.. :)

[forki]

Could you please send a pr which checks if the lical Paket exists and calls it instead of global one? Thst would be easy and non breaking change, right? Johny Woller Skovdal <notifications@github.com> schrieb am Sa., 9. Nov. 2019, 00:44:

…

Description Short version Some threads hang at dotnet paket --version and never moves on. This results in builds hanging because some projects never gets build; Visual Studio won't unload projects or ReSharper fails to load; possibly also causing other faulty behaviors. Longer version We've found that having the CPU max out sometimes causes dotnet paket --version commands to hang a process. It seems to be some kind of timing issue, but we are uncertain if the issue is in dotnet faulting on the output from paket, og if paket simply never exists correctly. The problem seems to only happen when using the new SDK project format (which is the one leveraging dotnet to begin with), as we do not have the same issue with our large Full Framework projects that have not yet been upgraded. Also, having anti virus monitoring file changes, as well as using ReSharper, seems to increase the likelyhood dramatically of it happening. We started noticing dotnet processes not closing down again after calling dotnet paket --version, and they seem to be what is blocking all the processes. Even more odd is, that when inspecting the process with Process Explorer <https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer> and going to the Threads tab, the process finally closes down. What exactly causes the process to close down from that inspection we have still not uncovered, as listing the threads of the process via PowerShell does not yield the same result. Repro steps 1. Have a large solution (in our case 230 projects) in Full Framework, but in the new SDK format. 2. Have ReSharper active in Visual Studio. 3. Work for a bit in Visual Studio, building and navigating around for 4-5 minutes. 4. Attempt to unload the solution Expected behavior Visual Studio should simply unload the solution. Actual behavior Unloads hangs at "Unloading project 0 of 230 ..." and never quits (have tried leaving it for multiple hours). In Process Explorer a process will be running with the command dotnet paket --version. Known workarounds Either: 1. Shut down the hanging process by going to the threads tab, or forcing the process to exit. Or: 2. Removing the part in the paket.targets file that actually calls dotnet paket --version, and replacing it with calling the local paket.exe always instead. This get overridden however when paket install/update is called, and must be corrected again, and there are no switches to prevent it from happening. Final thoughts This issue has been downright cripling for our development department, as everything slowed down to an almost standstill. Different developers saw the bug more or less frequently, but some had an unresponsive Visual Studio for 10-15 minutes at a time. We do not know, however, if the error is with dotnet instead of directly with paket, but it is only in combination of the two we have the error. The best solution, for us right now, I believe is to have some sort of flag, that would brevet paket to get asked all the time. I hope the issue makes somewhat sense, and that a more permanent solution can be found, rather than we manually fix the targets file, time and time again.. :) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#3705?email_source=notifications&email_token=AAAOANDZH5DSXZXETL6IH23QSX2UXA5CNFSM4JLAUE2KYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HYC56SQ>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAOANEW4I3FJZL6CAPATJTQSX2UXANCNFSM4JLAUE2A> .

[JohnyWS]

I hope so. But I can not be 100% sure when not completely understanding why it is prioritized as it is now? For instance, the global tools takes precedence over what is included in the repository, which makes me believe, that it will actually have an effect on existing installations to change that logic.

I will give at a try though, and we can discuss it around concrete code changes.

[forki]

Tbh I think local tools should always be preferred. Johny Woller Skovdal <notifications@github.com> schrieb am Sa., 9. Nov. 2019, 14:08:

…

I hope so. But I can not be 100% sure when not completely understanding why it is prioritized as it is now? For instance, the global tools takes precedence over what is included in the repository, which makes me believe, that it will actually have an effect on existing installations to change that logic. I will give at a try though, and we can discuss it around concrete code changes. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3705?email_source=notifications&email_token=AAAOANFAZ4RF5YKJJN4B4U3QS2Y5TA5CNFSM4JLAUE2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDUFYYY#issuecomment-552098915>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAOANEYVVBRZBNCV45AA3LQS2Y5TANCNFSM4JLAUE2A> .

[JohnyWS]

Then the next issue is how you determine if they are available or not. Delving deeper into it, a few things puzzles me with the target file.

E.g. why does windows need the .exe extension? In my understanding it is never needed? Windows will always attempt to run the .exe file if it exists, and you execute a command using no extension.

If not using the dotnet CLI approach, it seems, the fallback, if nothing is found, is the global path, but nothing to test for its presence. Perhaps that is where the distinction should be? If no local tools are found, use dotnet CLI if it works, if not, use global tool?

[JohnyWS]

This also seems to be an error:

<!-- no windows, try bootstrapper -->
<PaketExePath Condition=" '$(PaketExePath)' == '' AND '$(OS)' != 'Windows_NT' AND Exists('$(PaketBootStrapperExeDir)') ">$(PaketBootStrapperExeDir)paket.exe</PaketExePath>

Shouldn't "paket.exe" be included in the "Exists" test? I'll be changing that in the PR to begin with, and we'll see where it leads us.. :)

[forki]

I'm happy about all the help that we can get. It's extremely tricky to get those things working on all the platforms. Thanks for jumping in. Johny Woller Skovdal <notifications@github.com> schrieb am Sa., 9. Nov. 2019, 14:26:

…

This also seems to be an error: <!-- no windows, try bootstrapper --> <PaketExePath Condition=" '$(PaketExePath)' == '' AND '$(OS)' != 'Windows_NT' AND Exists('$(PaketBootStrapperExeDir)') ">$(PaketBootStrapperExeDir)paket.exe</PaketExePath> Shouldn't "paket.exe" be included in the "Exists" test? I'll be changing that in the PR to begin with, and we'll see where it leads us.. :) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3705?email_source=notifications&email_token=AAAOANBQDFZ3T33IMOL2JSDQS23CHA5CNFSM4JLAUE2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDUGD4A#issuecomment-552100336>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAOANFOPRBJCS625FFN3X3QS23CHANCNFSM4JLAUE2A> .

[JohnyWS]

You're welcome. Awesome product! :) And now I've forced 40 developers to use it here, so I'd better make sure it works for them. ;) I've opened #3706 that should be a good first step to solving it.