Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NamedPipeServer][Error ] Pipe failure: Pipe hasn't been connected yet #482

Closed
yvesdelvaux opened this issue Feb 10, 2017 · 63 comments
Closed

Comments

@yvesdelvaux
Copy link

Expected behavior

I can restart Docker without restart my pc.

Actual behavior

Docker doesn't restart and freeze.

Information

  • Diagnostic ID : 6C2349F8-BC33-4B0F-9D88-DA5E76C8BD03/2017-02-10_14-32-36
  • Reproductible case : Update the docker or restart it
  • host distribution and version (Windows version, build number, etc) Windows 10 14393

Steps to reproduce the behavior

  1. Then the Docker is running, change anything in configuration and restart it
  2. Docker is in the state "Docker is starting" but it doesn't start. It seems to be in freeze state.
@rn
Copy link
Contributor

rn commented Feb 10, 2017

Thanks for your report. Something seems to have gone wrong during the upgrade process. It's not clear what, but could you try:

Thanks

@rn
Copy link
Contributor

rn commented Feb 10, 2017

@mmisztal1980 reported the same error with exactly the same error message on #487 with diagnostics id 5239E3DB-D0D2-47AE-B158-5D03C5A0619A/2017-02-10_22-19-15

the error message is:

[16:35:11.136][NamedPipeServer][Error  ] Pipe failure: Pipe hasn't been connected yet.    at System.IO.Pipes.PipeStream.CheckReadOperations()
   at System.IO.Pipes.PipeStream.BeginRead(Byte[] buffer, Int32 offset, Int32 count, AsyncCallback callback, Object state)
   at System.IO.Stream.<>c.<BeginEndReadAsync>b__43_0(Stream stream, ReadWriteParameters args, AsyncCallback callback, Object state)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncTrim[TInstance,TArgs](TInstance thisRef, TArgs args, Func`5 beginMethod, Func`3 endMethod)
   at System.IO.Stream.BeginEndReadAsync(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Stream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
   at System.IO.Stream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count)
   at Docker.Core.Pipe.NamedPipeServer.<HandleRequestAsync>d__11.MoveNext()
[16:35:11.156][Notifications  ][Error  ] Binary stream '0' does not contain a valid BinaryHeader. Possible causes are invalid stream or object version change between serialization and deserialization.

@dvprao
Copy link

dvprao commented Feb 11, 2017

https://github.com/docker/for-win/issues
CAFC569A-DA17-4E8C-ABA5-D9AF5DFA1809/2017-02-10_16-08-08
error during connect: Get http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.26/containers/json: open //./pipe/docker_engine: The system cannot find the file specified. In the default daemon configuration on Windows, the docker client must be run elevated to connect. This error may also indicate that the docker daemon is not running.

Very frustrating for a newbie.

@rn
Copy link
Contributor

rn commented Feb 11, 2017

@dvprao thanks for the report. I have already filed on internal issue

@mmisztal1980
Copy link

@rneugeba It's impossible to use the "reset". All the options in the "Reset" pane are grayed out.

@mmisztal1980
Copy link

@rneugeba I've reinstalled docker for windows using the latest stable installer. Same issue.

@rn
Copy link
Contributor

rn commented Feb 11, 2017

@mmisztal1980 could you provide a diagnostics ID? thanks

@mmisztal1980
Copy link

Latest uploaded diagnostic: 5239E3DB-D0D2-47AE-B158-5D03C5A0619A/2017-02-11_10-55-40

@rn
Copy link
Contributor

rn commented Feb 11, 2017

@mmisztal1980 thanks for the quick response. Your issue seems different. could you file a separate issue? Thanks

@rn rn changed the title [Windows 10] Docker doesn't start [NamedPipeServer][Error ] Pipe failure: Pipe hasn't been connected yet Feb 11, 2017
@mmisztal1980
Copy link

mmisztal1980 commented Feb 11, 2017

I already did, but you closed it :D #487

@rn
Copy link
Contributor

rn commented Feb 11, 2017

Ah, sorry. Yes, you other logs under 5239E3DB-D0D2-47AE-B158-5D03C5A0619A/2017-02-10_22-19-15 do show this error. Sorry, for the confusion

@mmisztal1980
Copy link

No problem, I suppose we can reopen it and continue there?

@rn
Copy link
Contributor

rn commented Feb 11, 2017

no, your other logs contain the same error as the report on this issue. we need to look into why there is a backtrace and why subsequently nothing works anymore.

Did older versions work for you?
Did you install/update any software, in particular AV software?

@mmisztal1980
Copy link

mmisztal1980 commented Feb 11, 2017

It used to work up to a week ago, there was an update recently, but it didn't work by then.
My AV software is ESET NOD32, but I haven't made any updates to it's configuration recently.

From my perspective it stopped working silently, I haven't noticed exactly when and why.

@rn
Copy link
Contributor

rn commented Feb 11, 2017

Could you try disabling the AV software and see if it works?

Also, could you try to revert to 1.12.6 from here https://download.docker.com/win/stable/1.12.6.9655/InstallDocker.msi

and also 1.13.0 from here: https://download.docker.com/win/stable/1.13.0.9795/InstallDocker.msi

that would help us narrow down if it's something we changed.

Thanks

@mmisztal1980
Copy link

I've disabled the following features in NOD:

  • HIPS (host intrusion prevention system)
  • Real-time file system protection

No effect, docker is still "stuck" in "Docker is starting state". I'll try the older versions next.

@mmisztal1980
Copy link

mmisztal1980 commented Feb 11, 2017

v1.12.6 :

Docker seems to be stuck again. Last meaninfgul enrtry in the log suggests that it was trying to create the Hyper-V NAT switch.

Channel: Stable
Sha1: 258480fd147da7888a440ed7808d255c0fb15bb7
Started on: 2017/02/11 13:05:41.563
Resources: C:\Program Files\Docker\Docker\Resources
OS: Windows 10 Enterprise
Edition: Enterprise
Id: 1607
Build: 14393
BuildLabName: 14393.693.amd64fre.rs1_release.161220-1747
You can send feedback, including this log file, at https://github.com/docker/for-win/issues
[13:05:41.509][Tracking       ][Info   ] Crash report and usage statistics are enabled
[13:05:41.585][SegmentApi     ][Info   ] Usage statistic: appLaunched
[13:05:41.653][NamedPipeClient][Info   ] Sending Version()...
[13:05:41.653][NamedPipeClient][Info   ] Received response for Version
[13:05:41.669][SegmentApi     ][Info   ] Usage statistic: heartbeat
[13:05:36.255][BackendServer  ][Info   ] Started
[13:05:36.256][Moby           ][Info   ] Connecting...
[13:05:41.653][NamedPipeServer][Info   ] Version()
[13:05:41.653][NamedPipeServer][Info   ] Version done.
[13:05:41.872][Notifications  ][Info   ] Docker is starting...
[13:05:41.872][Updater        ][Info   ] Checking for updates on channel Stable...
[13:05:41.887][NamedPipeClient][Info   ] Sending Start(Docker.Core.Settings)...
[13:05:41.887][NamedPipeServer][Info   ] Start(Docker.Core.Settings)
[13:05:41.887][HyperVGuids    ][Info   ] Installing GUIDs...
[13:05:41.903][HyperVGuids    ][Info   ] GUIDs installed
[13:05:41.903][Firewall       ][Info   ] Opening ports for C:\Program Files\Docker\Docker\Resources\com.docker.proxy.exe...
[13:05:42.012][Updater        ][Info   ] We got a new version 10072 which is newer than 9655, asking user.
[13:05:42.012][SegmentApi     ][Info   ] Usage statistic: eventShowUpdateWindows
[13:05:42.327][Updater        ][Info   ] Check for updates done.
[13:05:42.984][Firewall       ][Info   ] Ports are opened
[13:05:42.986][HyperV         ][Info   ] Create
[13:05:42.993][PowerShell     ][Info   ] Run script with parameters: -Create True -SwitchSubnetAddress 10.0.75.0 -SwitchSubnetMaskSize 24 -CPUs 2 -Memory 2048 -IsoFile C:\Program Files\Docker\Docker\Resources\mobylinux.iso...
[13:05:45.144][HyperV         ][Info   ] Creating Switch: DockerNAT...
[13:06:14.242][SegmentApi     ][Info   ] Usage statistic: actionMenuSettings
[13:06:18.475][FeedbackSettings][Info   ] Open logs
[13:08:02.903][SegmentApi     ][Info   ] Usage statistic: actionMenuSettings
[13:09:17.835][FeedbackSettings][Info   ] Open logs
[13:10:24.675][SegmentApi     ][Info   ] Usage statistic: actionMenuFeedback
[13:10:26.063][FeedbackSettings][Info   ] Upload diagnostic
[13:10:26.066][NamedPipeClient][Info   ] Sending GetDebugInfo()...
[13:10:26.068][NamedPipeServer][Info   ] GetDebugInfo()
[13:10:37.591][NamedPipeServer][Info   ] GetDebugInfo done.
[13:10:37.605][NamedPipeClient][Info   ] Received response for GetDebugInfo
[13:10:37.612][Program        ][Info   ] Sending Bugsnag report e2ff260c-062c-4251-9040-c0ce6d2a7fb8...
[13:10:39.181][Program        ][Info   ] Bugsnag report e2ff260c-062c-4251-9040-c0ce6d2a7fb8 sent
[13:10:39.200][NamedPipeClient][Info   ] Sending DownloadVmLogs()...
[13:10:39.201][NamedPipeServer][Info   ] DownloadVmLogs()
[13:10:39.204][PowerShell     ][Info   ] Run script...

I'm attempting to upload a diagnostic but I'm not getting the UI to present me with the diagnostic's ID.

Hyper-V (MobyLinuxVM)

Last logs from the MobyLinuxVM originate from Jan29th.
'MobyLinuxVM' was shut down using the Shutdown Integration Component with parameters: Force = true, Reason = '' (Virtual machine ID AD7C949E-4A1C-4ACA-888E-119AED457A98)

Hyper-V (Networking)

Lat logs regarding the DockerNAT switch originate from Feb7th.
Switch deleted, name='F8FC6FFE-90F0-46E8-9AC9-91ADBC1A0426', friendly name='DockerNAT'.
I may have been reinstalling my NW drivers at that time.

@yvesdelvaux
Copy link
Author

Thanks for the answer. I have installed the new version manually. That's work for the moment. But the next update can fail if this issue is not solved.

I already have this problem at the previous update. The issue appears with the Windows container version of docker.

@jhagege
Copy link

jhagege commented Feb 12, 2017

+1

@rn
Copy link
Contributor

rn commented Feb 12, 2017

@yvesdelvaux thanks for getting back. This is weird...Do you have any AV software installed?

@cyberjoac can you provide a diagnostic ID?

@jhagege
Copy link

jhagege commented Feb 12, 2017

@rneugeba Just uploaded the diagnostic ID: 639B87D4-CE3B-4850-90DC-A38C83D12E6A/2017-02-12_16-01-57

Please note I can't see any VM in the Hyper-V manager..

@mmisztal1980
Copy link

@rneugeba while using v12, I was consistently unable to upload any diagnostics.
I've updated to v13.0. The main issue persists, however I'm now able to upload diagnostics again, so :
I've uploaded a new diagnostic : 5239E3DB-D0D2-47AE-B158-5D03C5A0619A/2017-02-12_17-37-41

@mmisztal1980
Copy link

mmisztal1980 commented Feb 12, 2017

Another observation:
v12: while navigating the panes of the docker for windows application I'm able to see controls' contents, such as Subnet Mask, Subnet Address etc. - can't see any content in v13.

@yvesdelvaux
Copy link
Author

@rneugeba Yes I have an AV (Trend Micro OfficeScan). I can't disable it. My firm has a strict security policy about it.

@rn
Copy link
Contributor

rn commented Feb 13, 2017

@cyberjoac thanks for the diagnostics ID. You indeed have the same issue as others.

@mmisztal1980 hmm, odd. Could you attach a screen shot from the 1.13 control panel? Thanks

@rn rn added the area/gui label Feb 13, 2017
@rn
Copy link
Contributor

rn commented Feb 14, 2017

Thanks a lot

@mmisztal1980
Copy link

@rneugeba
Interestingly, I found this in my device manager:

docker-v13-hyperv-issue2

@dineshgovindasamy
Copy link

Those are leaked VNIC's.

Do you have any proprietary teaming solution or LBFO team created on this host?

@dineshgovindasamy
Copy link

@mmisztal1980 @rneugeba Could you please also run the following cleanupscript and share us the logs?

https://github.com/Microsoft/Virtualization-Documentation/tree/live/windows-server-container-tools/CleanupContainerHostNetworking

Please run it with CleanupContainerHostNetworking.ps1 with the -ForceCleanup parameter, this will perform log collection and also cleanup those leaked NICS.

@mmisztal1980
Copy link

@dineshgovindasamy
I'm not sure what you mean? What is a LBFO or a teaming solution? You mean a VPN/SDN?

Logs attached. I've tried it twice, first using -Cleanup, the using -ForceDeleteAllSwitches

PreCleanupState_HWOLYOCW.zip
PreCleanupState_PSVKYOAH.zip
PostCleanupState_PSVKYOAH.zip

@rn
Copy link
Contributor

rn commented Feb 15, 2017

@JonCubed hit the same issue over at #417 (comment). The diagnostics ID is
41560D0B-E5AF-41B4-94BD-0C90C899004C/2017-02-11_22-59-1

@mmisztal1980
Copy link

mmisztal1980 commented Feb 15, 2017

@dineshgovindasamy
For clarity, I'm using zerotier SDN/VPN to stay connected to one of my networks. The iface is visible in the device manager screenshot.

I've been using it for about 1.5months. during that time I was able to use docker for windows.

@mmisztal1980
Copy link

@dineshgovindasamy @rneugeba
I've uninstalled zerotier and docker has started successfully.

@rn
Copy link
Contributor

rn commented Feb 18, 2017

@mmisztal1980 great to know. We recently had several issue with various AV/security softwares and looks like zerotier is another one to look out for

@rn
Copy link
Contributor

rn commented Feb 19, 2017

@yvesdelvaux do you have any AV/security software installed? And if so could you try disabling?

@JonCubed
Copy link

@rneugeba @dineshgovindasamy let me know if you need me to do anything to help debug this issue, love to get docker working again

@rn
Copy link
Contributor

rn commented Feb 20, 2017

@JonCubed Have you tried what I suggested here?Most importantly the Hyper-V test?

Also do you run any third party AV/FW software?

@kevinykuo
Copy link

OK guys this is what I did to resolve the issue on my end. Go to "Turn Windows features on or off", disable all Hyper-V related features, reboot, then Docker should ask if it can enable and reboot for you. Let it do that and see if it's fixed. If not I'd probably try manually re-enabling Hyper-V. Good luck.

@JonCubed
Copy link

JonCubed commented Feb 21, 2017

UI Test Diagnostic Id: 41560D0B-E5AF-41B4-94BD-0C90C899004C/2017-02-21_22-44-54

Hyper-V test fails on create with

New-NetIPAddress : The object already exists.
At C:\Program Files\Docker\Docker\resources\MobyLinux.ps1:164 char:23
+ ... rkAdapter | New-NetIPAddress -AddressFamily IPv4 -IPAddress $switchAd ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (MSFT_NetIPAddress:ROOT/StandardCimv2/MSFT_NetIPAddress) [New-NetIPAddress
   ], CimException
    + FullyQualifiedErrorId : Windows System Error 5010,New-NetIPAddress

as for av, just using windows defender

cc @rneugeba

@JonCubed
Copy link

JonCubed commented Mar 6, 2017

@rneugeba just following up on this, anything else I can do to help with this?

@JonCubed
Copy link

JonCubed commented Mar 6, 2017

I did some debugging turns out that I had a already had an assigned ip address for 10.0.75.1.

[DBG]: PS C:\WINDOWS\system32>> Get-NetIPAddress -IPAddress $switchAddress


IPAddress         : 10.0.75.1
InterfaceIndex    : 31
InterfaceAlias    : vEthernet (InternalNAT)
AddressFamily     : IPv4
Type              : Unicast
PrefixLength      : 24
PrefixOrigin      : Manual
SuffixOrigin      : Manual
AddressState      : Preferred
ValidLifetime     : Infinite ([TimeSpan]::MaxValue)
PreferredLifetime : Infinite ([TimeSpan]::MaxValue)
SkipAsSource      : False
PolicyStore       : ActiveStore

[DBG]: PS C:\Program Files\Docker\Docker\resources>> $networkAdapter | Format-List


Name                       : vEthernet (DockerNAT)
InterfaceDescription       : Hyper-V Virtual Ethernet Adapter #4
InterfaceIndex             : 112
MacAddress                 : 00-15-5D-4B-01-2E
MediaType                  : 802.3
PhysicalMediaType          : Unspecified
InterfaceOperationalStatus : Up
AdminStatus                : Up
LinkSpeed(Gbps)            : 10
MediaConnectionState       : Connected
ConnectorPresent           : False
DriverInformation          : Driver Date 2006-06-21 Version 10.0.14393.0 NDIS 6.50

this led me to ln#157 in MobyLinux.ps1

# Make sure the Switch has the right IP address
$networkAdapter = Get-NetAdapter | ? { $_.DeviceID -eq $vmNetAdapter.DeviceId }
if ($networkAdapter | Get-NetIPAddress -IPAddress $switchAddress -ea SilentlyContinue) {
        Write-Output "Using existing Switch IP address"
        return
}

if you remove -ea SilentlyContinue it throws the following error

Get-NetIPAddress : No matching MSFT_NetIPAddress objects found by CIM query for instances of the 
ROOT/StandardCimv2/MSFT_NetIPAddress class on the  CIM server: SELECT * FROM MSFT_NetIPAddress  WHERE 
((IPAddress LIKE '10.0.75.1')) AND ((InterfaceIndex = 112)) AND ((InterfaceAlias LIKE 'vEthernet 
(DockerNAT)')). Verify query parameters and retry.

you can see it is also filtering by InterfaceAlias vEthernet (DockerNAT) but mine is called vEthernet (InternalNAT)

not piping $networkAdapter would make the if statement work, not sure that is what your really want though.

To reproduce:

  1. .\MobyLixux.ps1 -Destroy
  2. New-NetIPAddress -AddressFamily IPv4 -IPAddress 10.0.75.1 -PrefixLength (24) -InterfaceAlias "vEthernet (InternalNAT)" -ea Stop
  3. .\MobyLinux.ps1 -Create

Workaround:

  1. .\MobyLixux.ps1 -Destroy
  2. Remove-NetIPAddress -IPAddress 10.0.75.1
  3. .\MobyLinux.ps1 -Create
  4. Restart docker for windows

cc @rneugeba

@ghost
Copy link

ghost commented Jun 5, 2017

This didn't work either. Docker for Windows client is frozen for about 30 seconds then fails with same pipe not readable error.

@JonCubed
Copy link

JonCubed commented Jun 8, 2017

@megamindbrian I had a problem with most recent version. I uninstalled it, removed all docker folders and server and virtual switch from hyper-v and then reinstalled and it worked. If you haven't tried this, give it a shot

@vvasilenok
Copy link

I have the same problem.
Diagnostic Id: CAB78893-6FDB-432D-B75F-E932445907AF/2017-07-20_10-15-51

[NamedPipeServer][Error ] Pipe failure: Pipe is broken. at System.IO.Pipes.PipeStream.WinIOError(Int32 errorCode)
at System.IO.Pipes.PipeStream.BeginWriteCore(Byte[] buffer, Int32 offset, Int32 count, AsyncCallback callback, Object state)
at System.IO.Stream.<>c.b__53_0(Stream stream, ReadWriteParameters args, AsyncCallback callback, Object state)
at System.Threading.Tasks.TaskFactory1.FromAsyncTrim[TInstance,TArgs](TInstance thisRef, TArgs args, Func5 beginMethod, Func`3 endMethod)
at System.IO.Stream.BeginEndWriteAsync(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.Stream.WriteAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
at System.IO.Stream.WriteAsync(Byte[] buffer, Int32 offset, Int32 count)
at Docker.Core.Pipe.NamedPipeServer.d__12.MoveNext() in C:\gopath\src\github.com\docker\pinata\win\src\Docker.Core\pipe\NamedPipeServer.cs:line 117

@tneward
Copy link

tneward commented Aug 15, 2017

Just to add data points--this is happening on my Win10 install as well just recently. Going to start trying some of the steps presented here, but if there's data I can provide to help track this down, lmk.

@ghost
Copy link

ghost commented Aug 15, 2017 via email

@tneward
Copy link

tneward commented Aug 15, 2017

Yeah, I tried that yesterday but the hanging behavior just started, so I'm thinking maybe after an uninstall things got left in a weird state. That's why I'm going to try some of the options mentioned in here as well as doing an uninstall/reinstall.

@rn
Copy link
Contributor

rn commented Mar 8, 2018

closing as the OP is pretty old

@docker-robott
Copy link
Collaborator

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked

@docker docker locked and limited conversation to collaborators Jun 20, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests