Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ssh causes system halt, BSOD after KB4489868 #3916

Open
raysuelzer opened this issue Mar 13, 2019 · 36 comments

Comments

Projects
None yet
@raysuelzer
Copy link

commented Mar 13, 2019

I know it says not to post BSOD, but this issue is impacting multiple users. When using ssh with the -A flag, or for some people when using any key, it causes a blue screen of death.

Details in the reddit thread: https://www.reddit.com/r/bashonubuntuonwindows/comments/b0o6v6/after_update_ssh_to_remote_hosts_cause_blue_screen/

@raysuelzer raysuelzer changed the title ssh causes system halt, BSDO after KB4489868 ssh causes system halt, BSOD after KB4489868 Mar 13, 2019

@chx

This comment has been minimized.

Copy link

commented Mar 13, 2019

I reported at #3915 I will close mine. I emailed minidumps to secure@microsoft.com got a canned reply "Thank you for contacting the Microsoft Security Response Center (MSRC). These types of support issues are not something that we can assist with directly. The appropriate page to contact support is available at:" blah blah blah.

@Brian-Perkins Brian-Perkins added the bug label Mar 14, 2019

@Brian-Perkins

This comment has been minimized.

Copy link
Collaborator

commented Mar 14, 2019

We were able to repro this internally and are working on a fix.

@xuan-w

This comment has been minimized.

Copy link

commented Mar 15, 2019

I have this problem too. I used ssh-agent and it lead to BSOD.

Good to know you are aware of the problem.

@jpaclcarneiro

This comment has been minimized.

Copy link

commented Mar 15, 2019

Also experiencing this problem when using ForwardAgent. Waiting for your fix!

@apendragon

This comment has been minimized.

Copy link

commented Mar 15, 2019

same issue on my side. It occurs only when I forward the agent (with -A option).

@zz9pzza

This comment has been minimized.

Copy link

commented Mar 16, 2019

As https://www.reddit.com/r/bashonubuntuonwindows/comments/b0o6v6/after_update_ssh_to_remote_hosts_cause_blue_screen/ says rolling back KB4489868 means I can continue to use my machine until there is another fix.

I really came to say thank you for making windows useable for developers and admins.

@achernyakevich-sc

This comment has been minimized.

Copy link

commented Mar 16, 2019

My experiments show that problem disappears when I have not used or turned off Agent Forwarding.

I had it turned on for all my SSH connections in the .ssh/config. So after commenting the following line in my config I have avoided BSOD:

ForwardAgent yes

Take into account that this setting could be defined for every host definition in your config so it could cause that some hosts are accessible and some will cause problem.

If you do not use config file then you still could reach to problem if you use Agent Forwarding by setting -A option for your ssh command. Just as temporary solution - you could avoid to use it.

Fortunately this and next week I don't need to work actively with hosts that need Agent Forwarding so I will just wait when guys will have fixed the root of the problem, prepare it as regular update and let us know in this thread that we can turn on Agent Forwarding again. :)

@tony1223

This comment has been minimized.

Copy link

commented Mar 17, 2019

This is a really annoying one.....the ssh is a core service.

@achernyakevich-sc

This comment has been minimized.

Copy link

commented Mar 22, 2019

@Brian-Perkins

We were able to repro this internally and are working on a fix.

8 days gone since you could reproduce the problem.

I would appreciate any news and estimation about when it could be fixed? Maybe you need any help?

@Brian-Perkins

This comment has been minimized.

Copy link
Collaborator

commented Mar 22, 2019

@achernyakevich-sc - as you are aware, shipping new bugs as part of updates is not ideal (where we find ourselves here), so there are a number of processes we have internally, designed to increase the quality of our update packages. We are navigating through those processes now. In this case, the issue is serious and the fix is fairly simple, so I am hopeful we can make the April release. We tend to avoid dates in these scenarios, as providing a date would imply more certainty than we actually have. I can say with certainty that everyone is aware that this needs fixed as quickly as possible.

@ggolda

This comment has been minimized.

Copy link

commented Mar 22, 2019

Is there any workaround that can help us to fix this issue? Agent forwarding is a part of my deployment process, sorry but we can not to wait until April, ssh is a core service.

May be its possible at least to build and install manually a beta prerelease version of this fix?

@crostine

This comment has been minimized.

Copy link

commented Mar 22, 2019

I have no workaround, in fact I wish there was something to just disable or ignore this update... (I'm using windows home), every 2 days this updates gets installed again and I have to manually remove it...
disabling the windows update service doesn't seem to do much as it re enables itself after a day or 2.

so I'll be removing this update until April I guess.

@tomforge

This comment has been minimized.

Copy link

commented Mar 22, 2019

Seems like you can pause updates in Settings > Windows Update > Advanced Settings

@ggolda

This comment has been minimized.

Copy link

commented Mar 22, 2019

@crostine hey, thanks for the suggestion. Can you pls explain or send a link how to revert this update? I think it will be useful for everyone!

@crostine

This comment has been minimized.

Copy link

commented Mar 22, 2019

I have tried many things, believe me.
in my windows home there is no such thing as pausing updates

I just tried this tool: https://support.microsoft.com/en-us/help/3073930/how-to-temporarily-prevent-a-driver-update-from-reinstalling-in-window
It looked like it was working, until I restarted the computer, then the update got installed anyway and well it's not hidden anymore even after removing it.

so maybe I can fake my connection to be a metered connection and prevent windows from downloading that way, at the cost of all the problems metered connections have.

@ggolda to remove the update
I go to windows update overview in settings
then click "view update history" almost at the top
then click "uninstall updates" at the top
it then opens a new window with the title "Installed Updates"
here you can uninstall the update by finding it in the list, selecting it and select uninstall.
and do the restart after the uninstall.

@achernyakevich-sc

This comment has been minimized.

Copy link

commented Mar 22, 2019

Dear @Brian-Perkins

so I am hopeful we can make the April release.

I know how is it when you develop and maintain complicated software solutions and what kind processes could be behind of it. Thank you very much for providing information about April release. We will keep fingers crossed to get it fixed as you plan. :)

@ggolda

This comment has been minimized.

Copy link

commented Mar 22, 2019

@crostine thanks!

@Po-wei

This comment has been minimized.

Copy link

commented Mar 24, 2019

My Laptop does not get this update, even if I press check for updates several times.
I think this update might be removed temporary.

@Po-wei

This comment has been minimized.

Copy link

commented Mar 24, 2019

@Brian-Perkins Hope April update will be better and more stable!

@kvietmeier

This comment has been minimized.

Copy link

commented Mar 26, 2019

Could this be related to:
https://community.mcafee.com/t5/Endpoint-Security-ENS/blue-screen-mfeepmpk-wsl-git/td-p/616729

I can't even open WSL from ConEmu or MobaXterm without a BSOD due to a pagefault by mfeepmpk. I don't even get to run SSH.

@kvietmeier

This comment has been minimized.

Copy link

commented Mar 26, 2019

The driver mentioned from the bugcheck is the Exploit Prevention driver (mfeepmpk.sys). A possible work-around would be to disable Exploit Prevention.

This works but is not an option for many of us working in restrictive corporate environments. I'll see if I can remove the update mentioned here.

@McJoppy

This comment has been minimized.

Copy link

commented Mar 26, 2019

To remove the patch I'm running wusa /uninstall /kb:4489868 via admin console.

Will try the pause updates to stop the re-installation too!

@mr-deamon

This comment has been minimized.

Copy link

commented Apr 1, 2019

The driver mentioned from the bugcheck is the Exploit Prevention driver (mfeepmpk.sys). A possible work-around would be to disable Exploit Prevention.

Worked for me! Can't tell if it was the windows-update or McAfee...

@KaspervdHeijden

This comment has been minimized.

Copy link

commented Apr 1, 2019

Uninstalling kb4489868 is just a temp fix; Windows Update will just reinstall it. Disabling WU just because of this will miss out on other potential key fixes. Very much hoping on a swift fix here!

@draycasejr

This comment has been minimized.

Copy link

commented Apr 5, 2019

@achernyakevich-sc how are you doing this?
I do not have a .ssh/config in any of my ubuntu directories so I can't see how this works.
I switched it off in /etc/ssh/ssh_config, but it didn't make a difference - it still blue screens when I try to ssh using Jsch via java. I couldn't find any way to turn off agent fowarding in Jsch.

My experiments show that problem disappears when I have not used or turned off Agent Forwarding.

I had it turned on for all my SSH connections in the .ssh/config. So after commenting the following line in my config I have avoided BSOD:

ForwardAgent yes

Take into account that this setting could be defined for every host definition in your config so it could cause that some hosts are accessible and some will cause problem.

If you do not use config file then you still could reach to problem if you use Agent Forwarding by setting -A option for your ssh command. Just as temporary solution - you could avoid to use it.

Fortunately this and next week I don't need to work actively with hosts that need Agent Forwarding so I will just wait when guys will have fixed the root of the problem, prepare it as regular update and let us know in this thread that we can turn on Agent Forwarding again. :)

@vasekboch

This comment has been minimized.

Copy link

commented Apr 6, 2019

I`ve upgraded to 1809 and the problem is gone.

@draycasejr

This comment has been minimized.

Copy link

commented Apr 6, 2019

Yes. 1809 fixes the BSOD, but now I can't ssh to a remote host. Regular Windows ssh works fine. Argh!!

WSL SCP works fine too. SMH

@chx

This comment has been minimized.

Copy link

commented Apr 6, 2019

I had SSH do that for me and a reboot fixed it. Go figure.

@McJoppy

This comment has been minimized.

Copy link

commented Apr 7, 2019

Just joined the club and installed 1809.

So far no issues.

@achernyakevich-sc

This comment has been minimized.

Copy link

commented Apr 8, 2019

Dear @draycasejr

@achernyakevich-sc how are you doing this?
I do not have a .ssh/config in any of my ubuntu directories so I can't see how this works.
I switched it off in /etc/ssh/ssh_config, but it didn't make a difference - it still blue screens when I try to ssh using Jsch via java. I couldn't find any way to turn off agent fowarding in Jsch.

SSH client config file usually is located at ~/.ssh/config, If you cant find it then you have no special config. But this is used only for ssh utility.

So in your case as you use Java solution this config file even if it exists will not be applied. You need to find a way how to disable AgentForwarding from inside of Java utility.

Though it looks as not actual anymore. I have just installed Features Update 1809 and problem is gone completely. So the simplest way for you could be just install latest updates.

Finally, @Brian-Perkins - it looks this issues could be marked as resolved and get closed.

Thanks to everybody who helped to get it fixed! :)

@maym2104

This comment has been minimized.

Copy link

commented Apr 9, 2019

The BSOD I got were from using rsync and scp on Ubuntu 18.04; update seemed to have fixed it as well.

@Brian-Perkins

This comment has been minimized.

Copy link
Collaborator

commented Apr 9, 2019

To address this on RS3 (version 1709, build 16299.*), apply KB4493441. If you are able to upgrade to a newer release, that is also a good solution as there are new WSL features in every release.

@Narimm

This comment has been minimized.

Copy link

commented Apr 10, 2019

So according to https://support.microsoft.com/en-us/help/4493464/windows-10-update-kb4493464 a patch was rolled out to address this and I can confirm on my system that it no longer blue screens when I enabled agent forwarding in the config....that being said its NOT actually agent forwarding at all..

To be sure prior to the update I tried to agent forward - and it blue screens I then updated and tried again nad we had no blue screen = however no I have no agent forward despite using -A to force it.

@Brian-Perkins

This comment has been minimized.

Copy link
Collaborator

commented Apr 10, 2019

@Narimm - now you are likely running into #3183 which unfortunately was not fixed until a couple of releases later RS5/1809. Prior to this fix we did not support closing one end of a unix socket and still getting credentials from the other, so it would only work when you got lucky racing with the close. That race window has narrowed with the latest RS3 packages, so the unfortunate side-effect is you are more apt to hit this issue.

@Narimm

This comment has been minimized.

Copy link

commented Apr 11, 2019

Yeah thanks I just installed RS5/1809 and confirm via the verbose ssh logs that forwarding now works as expected - I would suggest this issue be marked as resolved and closed I would say you could close #3183 based on my finding as well...

I can confirm forwarding was not working via verbose logs prior to [ RS5/1809.]
(https://blogs.windows.com/windowsexperience/2018/10/02/how-to-get-the-windows-10-october-2018-update)
However I cant find mention in any KB were a fix was applied in this update

However it has fixed all the forwarding issues. and blue screens

@ddukic

This comment has been minimized.

Copy link

commented May 7, 2019

Working after latest win update May 2019 version 1809.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.