New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failure on running linux containers on windows 10 (beta 31) #267

Closed
Ncage1974 opened this Issue Nov 30, 2016 · 41 comments

Comments

Projects
None yet
@Ncage1974
Copy link

Ncage1974 commented Nov 30, 2016

I was eagerly awaiting Beta 31 because it fixed the following issue
#54

Unfortunately its more of the same. When i open up hyper-v i can see 'MobyLinuxVM' is constantly hogging about 25% of the CPU and restarting and doing it again and again until docker fails. The error message this time though is different:

Docker hv-sock proxy (vsudd) is not reachable
at Docker.Backend.ContainerEngine.Linux.ConnectToVsud(TaskCompletionSource`1 vmId)
at Docker.Backend.ContainerEngine.Linux.DoStart(Settings settings)
at Docker.Backend.ContainerEngine.Linux.Start(Settings settings)
at Docker.Core.Pipe.NamedPipeServer.<>c__DisplayClass8_0.b__0(Object[] parameters)
at Docker.Core.Pipe.NamedPipeServer.RunAction(String action, Object[] parameters)

I'm running docker inside VMware workstation 12.5.2 on top of windows 10 professional with virtualization enabled in the bios of the VM.

Please note that i was about to use the workaround in the previously listed issue to get things working.

Here is the log:

Version: 1.13.0-rc2-beta31 (9123)
Channel: Beta
Sha1: de6f6c2e8d5ff4561b88bc212b9e2833b8684e4c
Started on: 2016/11/30 07:47:03.928
Resources: C:\Program Files\Docker\Docker\Resources
OS: Windows 10 Enterprise
Edition: Enterprise
Id: 1607
Build: 14393
BuildLabName: 14393.447.amd64fre.rs1_release_inmarket.161102-0100
File: C:\Users\waldrbr\AppData\Local\Docker\log.txt
CommandLine: "C:\Program Files\Docker\Docker\Docker for Windows.exe" 
You can send feedback, including this log file, at https://github.com/docker/for-win/issues
[07:47:04.194][GUI            ][Info   ] Starting...
[07:47:05.272][Tracking       ][Info   ] Crash report and usage statistics are enabled
[07:47:05.272][SegmentApi     ][Info   ] Usage statistic: appLaunched
[07:47:05.601][NamedPipeClient][Info   ] Sending Version()...
[07:47:05.616][NamedPipeClient][Info   ] Received response for Version
[07:47:05.616][SegmentApi     ][Info   ] Usage statistic: heartbeat
[07:46:58.069][BackendServer  ][Info   ] Started
[07:47:05.616][NamedPipeServer][Info   ] Version()
[07:47:05.616][NamedPipeServer][Info   ] Version done in 00:00:00.
[07:47:05.819][NamedPipeClient][Info   ] Sending Start(Docker.Core.Settings)...
[07:47:05.819][Updater        ][Info   ] Checking for updates on channel Beta...
[07:47:05.819][NamedPipeServer][Info   ] Start(Docker.Core.Settings)
[07:47:05.834][PowerShell     ][Info   ] Run script...
[07:47:05.834][PowerShell     ][Info   ] Creating a Runspace Pool...
[07:47:06.038][PowerShell     ][Info   ] Runspace Pool created (Min=1, Max=2)
[07:47:06.475][Updater        ][Info   ] Local build 9123 is as good as the remote 9123 on channel Beta
[07:47:09.819][PowerShell     ][Info   ] Run script...
[07:47:13.382][WindowsDockerDaemon][Info   ] Starting C:\Program Files\Docker\Docker\Resources\dockerd.exe -H npipe:////./pipe/docker_engine_windows
[07:47:13.382][WindowsDockerDaemon][Info   ] Started
[07:47:13.382][ApiProxy       ][Info   ] Starting C:\Program Files\Docker\Docker\Resources\com.docker.proxy.exe -daemonNamedPipe=\\.\pipe\docker_engine_windows
[07:47:13.397][ApiProxy       ][Info   ] Started
[07:47:13.467][ApiProxy       ][Info   ] docker proxy (on deprecated port): ready
[07:47:13.472][ApiProxy       ][Info   ] docker proxy: ready
[07:47:13.511][ApiProxy       ][Info   ] proxy >> GET /_ping
[07:47:13.511][ApiProxy       ][Info   ] Dial name pipe  \\.\pipe\docker_engine_windows
[07:47:13.512][ApiProxy       ][Info   ] open \\.\pipe\docker_engine_windows: The system cannot find the file specified.
[07:47:13.513][ApiProxy       ][Info   ] proxy >> GET /v1.25/containers/json
[07:47:13.513][ApiProxy       ][Info   ] Dial name pipe  \\.\pipe\docker_engine_windows
[07:47:13.513][ApiProxy       ][Info   ] open \\.\pipe\docker_engine_windows: The system cannot find the file specified.
[07:47:13.530][WindowsDockerDaemon][Info   ] time="2016-11-30T07:47:13.528857700-06:00" level=info msg="Windows default isolation mode: hyperv" 
[07:47:13.533][WindowsDockerDaemon][Info   ] time="2016-11-30T07:47:13.533244300-06:00" level=info msg="[graphdriver] using prior storage driver: windowsfilter" 
[07:47:13.569][WindowsDockerDaemon][Info   ] time="2016-11-30T07:47:13.568371500-06:00" level=info msg="Graph migration to content-addressability took 0.00 seconds" 
[07:47:13.569][WindowsDockerDaemon][Info   ] time="2016-11-30T07:47:13.569468100-06:00" level=info msg="Loading containers: start." 
[07:47:13.832][WindowsDockerDaemon][Info   ] time="2016-11-30T07:47:13.832525800-06:00" level=info msg="Loading containers: done." 
[07:47:13.835][WindowsDockerDaemon][Info   ] time="2016-11-30T07:47:13.835502100-06:00" level=info msg="Daemon has completed initialization" 
[07:47:13.835][WindowsDockerDaemon][Info   ] time="2016-11-30T07:47:13.835502100-06:00" level=info msg="Docker daemon" commit=1f9b3ef graphdriver=windowsfilter version=1.13.0-rc2 
[07:47:13.847][WindowsDockerDaemon][Info   ] time="2016-11-30T07:47:13.847509300-06:00" level=info msg="API listen on //./pipe/docker_engine_windows" 
[07:47:17.895][DockerDaemonChecker][Error  ] Docker daemon is not running
[07:47:18.030][ApiProxy       ][Info   ] proxy >> GET /_ping
[07:47:18.031][ApiProxy       ][Info   ] Dial name pipe  \\.\pipe\docker_engine_windows
[07:47:18.031][ApiProxy       ][Info   ] Successfully dialed name pipe \\.\pipe\docker_engine_windows
[07:47:18.038][ApiProxy       ][Info   ] proxy << GET /_ping
[07:47:18.039][ApiProxy       ][Info   ] proxy >> GET /v1.25/containers/json
[07:47:18.039][ApiProxy       ][Info   ] Dial name pipe  \\.\pipe\docker_engine_windows
[07:47:18.039][ApiProxy       ][Info   ] Successfully dialed name pipe \\.\pipe\docker_engine_windows
[07:47:18.040][ApiProxy       ][Info   ] proxy << GET /v1.25/containers/json
[07:47:18.060][NamedPipeClient][Info   ] Received response for Start
[07:47:18.057][DockerDaemonChecker][Info   ] Docker daemon is running
[07:47:18.058][NamedPipeServer][Info   ] Start done in 00:00:12.2375575.
[07:47:18.112][Notifications  ][Info   ] Docker is running
[07:47:18.112][SegmentApi     ][Info   ] Usage statistic: appRunning
[07:47:18.139][Actions        ][Debug  ] Start took 00:00:12.3510946
[07:47:34.006][SegmentApi     ][Info   ] Usage statistic: actionMenuSwitchLinuxDaemon
[07:47:34.012][Notifications  ][Info   ] Docker is switching...
[07:47:34.050][SegmentApi     ][Info   ] Usage statistic: actionSettingsChanged
[07:47:34.050][NamedPipeClient][Info   ] Sending Start(Docker.Core.Settings)...
[07:47:34.051][NamedPipeServer][Info   ] Start(Docker.Core.Settings)
[07:47:34.056][HyperV         ][Info   ] Create
[07:47:34.058][HyperVGuids    ][Info   ] Installing GUIDs...
[07:47:34.062][HyperVGuids    ][Info   ] GUIDs installed
[07:47:34.064][PowerMode      ][Info   ] Start
[07:47:34.071][PowerShell     ][Info   ] Run script with parameters: -Create True -SwitchSubnetAddress 10.0.75.0 -SwitchSubnetMaskSize 24 -CPUs 2 -Memory 2048 -IsoFile C:\Program Files\Docker\Docker\Resources\mobylinux.iso...
[07:47:34.074][Firewall       ][Info   ] Opening ports for C:\Program Files\Docker\Docker\Resources\com.docker.proxy.exe...
[07:47:34.242][HyperV         ][Info   ] Script started at 07:47:34.219
[07:47:35.996][Firewall       ][Info   ] Ports are opened
[07:47:36.877][HyperV         ][Info   ] Module loaded at 07:47:36.877
[07:47:38.244][HyperV         ][Info   ] Using existing Switch: DockerNAT
[07:47:39.021][HyperV         ][Info   ] Using existing Switch IP address
[07:47:41.151][HyperV         ][Info   ] Setting CPUs to 2 and Memory to 2048 MB
[07:47:43.152][HyperV         ][Info   ] Connect Internal Switch DockerNAT
[07:47:45.111][HyperV         ][Info   ] VM created.
[07:47:45.112][HyperV         ][Debug  ] [create] took 00:00:11.0541526 to run
[07:47:45.112][PowerShell     ][Info   ] Run script '(Get-VM MobyLinuxVM).Id.Guid'...
[07:47:45.215][DataKit        ][Info   ] Starting C:\Program Files\Docker\Docker\Resources\com.docker.db.exe --url hyperv-connect://c8f7b1ed-d4cf-48a2-aefd-048cd426cf25,\\.\pipe\dockerDataBase --color never
[07:47:45.221][DataKit        ][Info   ] Started
[07:47:45.284][DataKit        ][Info   ] Starting com.docker.db.exe 0.7.0-34-g6f04213 ...
[07:47:45.303][DataKit        ][Info   ] com.docker.db.exe: accepted a new connection on \\.\pipe\dockerDataBase
[07:47:45.307][Database       ][Info   ] Writing com.docker.driver.amd64-linux/net/config=static
[07:47:45.307][Database       ][Info   ] Writing com.docker.driver.amd64-linux/net/address=10.0.75.2
[07:47:45.307][Database       ][Info   ] Writing com.docker.driver.amd64-linux/net/netmask=255.255.255.0
[07:47:45.307][Database       ][Info   ] Writing com.docker.driver.amd64-linux/net/gateway=10.0.75.1
[07:47:45.307][Database       ][Info   ] Writing com.docker.driver.amd64-linux/network=hybrid
[07:47:45.307][Database       ][Info   ] Writing com.docker.driver.amd64-linux/etc/docker/daemon.json={"registry-mirrors":[],"insecure-registries":[]}
[07:47:45.307][Database       ][Info   ] Writing com.docker.driver.amd64-linux/etc/sysctl.conf=
[07:47:45.307][Database       ][Info   ] Writing com.docker.driver.amd64-linux/native/port-forwarding=true
[07:47:45.366][DataKit        ][Info   ] com.docker.db.exe: accepted a new connection on \\.\pipe\dockerDataBase
[07:47:45.367][DataKit        ][Info   ] com.docker.db.exe: Using protocol TwoThousand msize 16384
[07:47:47.131][Database       ][Info   ] Generating CA Cert Bundle
[07:47:47.155][Database       ][Info   ] CA Cert Bundle Generated
[07:47:47.155][Database       ][Info   ] Successfully wrote 8 value(s)
[07:47:47.156][Database       ][Info   ] Writing com.docker.driver.amd64-linux/etc/ssl/certs/ca-certificates.crt=C:\WINDOWS\TEMP\tmp418E.tmp
[07:47:47.204][DataKit        ][Info   ] com.docker.db.exe: accepted a new connection on \\.\pipe\dockerDataBase
[07:47:47.204][DataKit        ][Info   ] com.docker.db.exe: Using protocol TwoThousand msize 16384
[07:47:47.311][Database       ][Info   ] Successfully wrote value
[07:47:47.320][Moby           ][Info   ] Connecting...
[07:47:47.321][Moby           ][Error  ] Failed to read Moby's logs: The operation has timed out.
[07:47:47.325][HyperV         ][Info   ] Start
[07:47:47.325][PowerShell     ][Info   ] Run script with parameters: -Start True...
[07:47:47.344][PowerShell     ][Info   ] Run script '$(Find-NetRoute -RemoteIPAddress 8.8.8.8).InterfaceIndex[0]'...
[07:47:47.357][HyperV         ][Info   ] Script started at 07:47:47.357
[07:47:47.367][HyperV         ][Info   ] Module loaded at 07:47:47.367
[07:47:47.385][HyperV         ][Info   ] Starting VM MobyLinuxVM...
[07:47:48.965][Database       ][Info   ] Writing com.docker.driver.amd64-linux/slirp/dns=nameserver 10.2.1.5
nameserver 10.2.1.6
search TRS.STATE.IL.US
[07:47:49.025][DataKit        ][Info   ] com.docker.db.exe: accepted a new connection on \\.\pipe\dockerDataBase
[07:47:49.026][DataKit        ][Info   ] com.docker.db.exe: Using protocol TwoThousand msize 16384
[07:47:49.161][Database       ][Info   ] Successfully wrote 1 value(s)
[07:47:49.213][VpnKit         ][Info   ] Starting C:\Program Files\Docker\Docker\Resources\com.docker.slirp.exe --ethernet hyperv-connect://c8f7b1ed-d4cf-48a2-aefd-048cd426cf25 --port hyperv-connect://c8f7b1ed-d4cf-48a2-aefd-048cd426cf25 --db \\.\pipe\dockerDataBase --debug --diagnostics \\.\pipe\dockerVpnKitDiagnostics
[07:47:49.216][VpnKit         ][Info   ] Started
[07:47:57.629][VpnKit         ][Info   ] com.docker.slirp.exe: Logging to stdout (stdout:true DEBUG:false)
[07:47:57.629][VpnKit         ][Info   ] com.docker.slirp.exe: Setting handler to ignore all SIGPIPE signals
[07:47:57.629][VpnKit         ][Info   ] com.docker.slirp.exe: vpnkit version %VERSION% with hostnet version  %HOSTNET_PINNED% uwt version 0.0.3 hvsock version 0.11.1 %HVSOCK_PINNED%
[07:47:57.629][VpnKit         ][Info   ] com.docker.slirp.exe: starting port forwarding server on port_control_url:hyperv-connect://c8f7b1ed-d4cf-48a2-aefd-048cd426cf25 vsock_path:
[07:47:57.629][VpnKit         ][Info   ] com.docker.slirp.exe: connecting to c8f7b1ed-d4cf-48a2-aefd-048cd426cf25:0B95756A-9985-48AD-9470-78E060895BE7
[07:47:57.630][VpnKit         ][Debug  ] com.docker.slirp.exe: Waiting for connections on socket hyperv-connect://c8f7b1ed-d4cf-48a2-aefd-048cd426cf25
[07:47:57.630][VpnKit         ][Info   ] com.docker.slirp.exe: connecting to c8f7b1ed-d4cf-48a2-aefd-048cd426cf25:30D48B34-7D27-4B0B-AAAF-BBBED334DD59
[07:47:57.631][VpnKit         ][Debug  ] com.docker.slirp.exe: Waiting for connections on socket hyperv-connect://c8f7b1ed-d4cf-48a2-aefd-048cd426cf25
[07:47:57.631][VpnKit         ][Error  ] com.docker.slirp.exe: While watching /etc/resolv.conf: ENOENT
[07:47:57.631][VpnKit         ][Info   ] com.docker.slirp.exe: hosts file has bindings for 
[07:48:30.060][HyperV         ][Debug  ] [start] took 00:00:42.7353053 to run
[07:48:30.062][Linux          ][Info   ] Trying to connect to vsud...
[07:48:31.702][Moby           ][Info   ] Connected
[07:48:32.467][Moby           ][Info   ] done.
[07:48:32.468][Moby           ][Info   ] Booting the kernel.
[07:48:32.485][Moby           ][Info   ] Linux version 4.8.10-moby (root@1d811a9194c4) (gcc version 5.3.0 (Alpine 5.3.0) ) #1 SMP Mon Nov 21 22:02:05 UTC 2016
[07:48:32.507][Moby           ][Info   ] Command line: earlyprintk=ttyS0,115200 console=ttyS0,115200 mobyplatform=windows rootdelay=300 noautodetect
[07:48:32.507][Moby           ][Info   ] 
[07:48:32.520][Moby           ][Info   ] Disabled fast string operations
[07:48:32.540][Moby           ][Info   ] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[07:48:32.564][Moby           ][Info   ] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[07:48:32.584][Moby           ][Info   ] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[07:48:32.589][Moby           ][Info   ] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[07:48:32.606][Moby           ][Info   ] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[07:48:32.621][Moby           ][Info   ] x86/fpu: Using 'eager' FPU context switches.
[07:48:32.635][Moby           ][Info   ] e820: BIOS-provided physical RAM map:
[07:48:32.644][Moby           ][Info   ] BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable
[07:48:32.647][Moby           ][Info   ] BIOS-e820: [mem 0x0000000000100000-0x000000007eeedfff] usable
[07:48:32.656][Moby           ][Info   ] BIOS-e820: [mem 0x000000007eeee000-0x000000007eef1fff] ACPI data
[07:48:32.669][Moby           ][Info   ] BIOS-e820: [mem 0x000000007eef2000-0x000000007ef1afff] reserved
[07:48:32.686][Moby           ][Info   ] BIOS-e820: [mem 0x000000007ef1b000-0x000000007ff9afff] usable
[07:48:32.698][Moby           ][Info   ] BIOS-e820: [mem 0x000000007ff9b000-0x000000007fff2fff] reserved
[07:48:32.711][Moby           ][Info   ] BIOS-e820: [mem 0x000000007fff3000-0x000000007fffafff] ACPI data
[07:48:32.725][Moby           ][Info   ] BIOS-e820: [mem 0x000000007fffb000-0x000000007fffefff] ACPI NVS
[07:48:32.739][Moby           ][Info   ] BIOS-e820: [mem 0x000000007ffff000-0x000000007fffffff] usable
[07:48:32.754][Moby           ][Info   ] bootconsole [earlyser0] enabled
[07:48:32.771][Moby           ][Info   ] NX (Execute Disable) protection: active
[07:48:32.797][Moby           ][Info   ] efi: EFI v2.31 by EDK II
[07:48:32.825][Moby           ][Info   ] efi:  ACPI=0x7fffa000  ACPI 2.0=0x7fffa014  SMBIOS=0x7fff2000  PROP=0x4fa13d8  MEMATTR=0x7f524d18 
[07:48:32.836][Moby           ][Info   ] SMBIOS 2.4 present.
[07:48:32.860][Moby           ][Info   ] Hypervisor detected: Microsoft HyperV
[07:48:32.878][Moby           ][Info   ] HyperV: features 0x2e7f, hints 0x2c2c
[07:48:32.898][Moby           ][Info   ] HyperV: LAPIC Timer Frequency: 0x1e8480
[07:48:32.917][Moby           ][Info   ] clocksource: hyperv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x24e6a1710, max_idle_ns: 440795202120 ns
[07:48:32.941][Moby           ][Info   ] tsc: Marking TSC unstable due to running on Hyper-V
[07:48:32.950][Moby           ][Info   ] e820: last_pfn = 0x80000 max_arch_pfn = 0x400000000
[07:48:32.955][Moby           ][Info   ] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WC  UC- WT  
[07:48:32.960][Moby           ][Info   ] RAMDISK: [mem 0x7882c000-0x7b87dfff]
[07:48:32.964][Moby           ][Info   ] ACPI: Early table checksum verification disabled
[07:48:32.994][Moby           ][Info   ] ACPI: RSDP 0x000000007FFFA014 000024 (v02 VRTUAL)
[07:48:33.005][Moby           ][Info   ] ACPI: XSDT 0x000000007FFF90E8 00005C (v01 VRTUAL MICROSFT 00000001 MSFT 00000001)
[07:48:33.028][Moby           ][Info   ] ACPI: FACP 0x000000007FFF7000 0000F4 (v04 VRTUAL MICROSFT 00000001 MSFT 00000001)
[07:48:33.046][Moby           ][Info   ] ACPI: DSDT 0x000000007EEEF000 00256F (v02 MSFTVM DSDT01   00000001 MSFT 04000000)
[07:48:33.073][Moby           ][Info   ] ACPI: FACS 0x000000007FFFE000 000040
[07:48:33.089][Moby           ][Info   ] ACPI: APIC 0x000000007FFF8000 0007D2 (v02 VRTUAL MICROSFT 00000001 MSFT 00000001)
[07:48:33.102][Moby           ][Info   ] ACPI: OEM0 0x000000007FFF6000 000064 (v01 VRTUAL MICROSFT 00000001 MSFT 00000001)
[07:48:33.115][Moby           ][Info   ] ACPI: WAET 0x000000007FFF5000 000028 (v01 VRTUAL MICROSFT 00000001 MSFT 00000001)
[07:48:33.144][Moby           ][Info   ] ACPI: OEM1 0x000000007FFF4000 00009E (v02 MSFTVM UARTS    00000001 MSFT 04000000)
[07:48:33.161][Moby           ][Info   ] ACPI: OEM2 0x000000007FFF3000 0002E2 (v02 MSFTVM VTPM2    00000001 MSFT 04000000)
[07:48:33.174][Moby           ][Info   ] ACPI: SRAT 0x000000007EEEE000 000190 (v02 VRTUAL MICROSFT 00000001 MSFT 00000001)
[07:48:33.206][Moby           ][Info   ] Zone ranges:
[07:48:33.223][Moby           ][Info   ]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[07:48:33.237][Moby           ][Info   ]   DMA32    [mem 0x0000000001000000-0x000000007fffffff]
[07:48:33.253][Moby           ][Info   ]   Normal   empty
[07:48:33.265][Moby           ][Info   ] Movable zone start for each node
[07:48:33.288][Moby           ][Info   ] Early memory node ranges
[07:48:33.302][Moby           ][Info   ]   node   0: [mem 0x0000000000001000-0x000000000009ffff]
[07:48:33.332][Moby           ][Info   ]   node   0: [mem 0x0000000000100000-0x000000007eeedfff]
[07:48:33.344][Moby           ][Info   ]   node   0: [mem 0x000000007ef1b000-0x000000007ff9afff]
[07:48:33.362][Moby           ][Info   ]   node   0: [mem 0x000000007ffff000-0x000000007fffffff]
[07:48:33.376][Moby           ][Info   ] Initmem setup node 0 [mem 0x0000000000001000-0x000000007fffffff]
[07:48:33.391][Moby           ][Info   ] ACPI: PM-Timer IO Port: 0x408
[07:48:33.400][Moby           ][Info   ] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[07:48:33.408][Moby           ][Info   ] IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
[07:48:33.426][Moby           ][Info   ] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[07:48:33.438][Moby           ][Info   ] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[07:48:33.444][Moby           ][Info   ] Using ACPI (MADT) for SMP configuration information
[07:48:33.461][Moby           ][Info   ] smpboot: 240 Processors exceeds NR_CPUS limit of 128
[07:48:33.485][Moby           ][Info   ] smpboot: Allowing 128 CPUs, 126 hotplug CPUs
[07:48:33.495][Moby           ][Info   ] e820: [mem 0x80000000-0xffffffff] available for PCI devices
[07:48:33.507][Moby           ][Info   ] Booting paravirtualized kernel on bare hardware
[07:48:33.527][Moby           ][Info   ] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[07:48:33.546][Moby           ][Info   ] setup_percpu: NR_CPUS:128 nr_cpumask_bits:128 nr_cpu_ids:128 nr_node_ids:1
[07:48:33.561][Moby           ][Info   ] percpu: Embedded 35 pages/cpu @ffff880076800000 s105048 r8192 d30120 u262144
[07:48:33.568][Moby           ][Info   ] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 515799
[07:48:33.578][Moby           ][Info   ] Kernel command line: earlyprintk=ttyS0,115200 console=ttyS0,115200 mobyplatform=windows rootdelay=300 noautodetect
[07:48:33.578][Moby           ][Info   ] 
[07:48:33.592][Moby           ][Info   ] log_buf_len individual max cpu contribution: 4096 bytes
[07:48:33.600][Moby           ][Info   ] log_buf_len total cpu_extra contributions: 520192 bytes
[07:48:33.614][Moby           ][Info   ] log_buf_len min size: 131072 bytes
[07:48:33.632][Moby           ][Info   ] log_buf_len: 1048576 bytes
[07:48:33.640][Moby           ][Info   ] early log buf free: 122776(93%)
[07:48:33.651][Moby           ][Info   ] PID hash table entries: 4096 (order: 3, 32768 bytes)
[07:48:33.672][Moby           ][Info   ] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
[07:48:33.679][Moby           ][Info   ] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
[07:48:33.698][Moby           ][Info   ] Memory: 1944256K/2096184K available (8442K kernel code, 1394K rwdata, 2848K rodata, 1372K init, 588K bss, 151928K reserved, 0K cma-reserved)
[07:48:33.710][Moby           ][Info   ] Hierarchical RCU implementation.
[07:48:33.720][Moby           ][Info   ] 	Build-time adjustment of leaf fanout to 64.
[07:48:33.727][Moby           ][Info   ] Using NULL legacy PIC
[07:48:33.733][Moby           ][Info   ] NR_IRQS:8448 nr_irqs:1448 0
[07:48:33.754][Moby           ][Info   ] Console: colour dummy device 80x25
[07:48:33.772][Moby           ][Info   ] console [ttyS0] enabled
[07:48:33.786][Moby           ][Info   ] console [ttyS0] enabled
[07:48:33.804][Moby           ][Info   ] bootconsole [earlyser0] disabled
[07:48:33.812][Moby           ][Info   ] bootconsole [earlyser0] disabled
[07:48:34.488][Moby           ][Info   ] tsc: Fast TSC calibration failed
[07:48:34.494][Moby           ][Info   ] tsc: Unable to calibrate against PIT
[07:48:34.508][Moby           ][Info   ] tsc: HPET/PMTIMER calibration failed
[07:49:30.408][Linux          ][Error  ] Failed to start: Docker hv-sock proxy (vsudd) is not reachable. Let's retry
[07:49:30.408][PowerShell     ][Info   ] Run script...
[07:49:33.535][HyperV         ][Info   ] Hyper-V is running
[07:49:33.536][HyperV         ][Info   ] Create
[07:49:33.536][PowerShell     ][Info   ] Run script with parameters: -Create True -SwitchSubnetAddress 10.0.75.0 -SwitchSubnetMaskSize 24 -CPUs 2 -Memory 2048 -IsoFile C:\Program Files\Docker\Docker\Resources\mobylinux.iso...
[07:49:33.566][HyperV         ][Info   ] Script started at 07:49:33.566
[07:49:33.574][HyperV         ][Info   ] Module loaded at 07:49:33.574
[07:49:34.879][HyperV         ][Info   ] Using existing Switch: DockerNAT
[07:49:35.609][HyperV         ][Info   ] Using existing Switch IP address
[07:49:35.715][HyperV         ][Info   ] VM MobyLinuxVM is Running. Cannot change its settings.
[07:49:35.722][HyperV         ][Debug  ] [create] took 00:00:02.1865781 to run
[07:49:35.722][PowerMode      ][Info   ] Stop
[07:49:35.725][HyperV         ][Info   ] Stop
[07:49:35.725][PowerShell     ][Info   ] Run script with parameters: -Stop True...
[07:49:35.740][HyperV         ][Info   ] Script started at 07:49:35.739
[07:49:35.745][HyperV         ][Info   ] Module loaded at 07:49:35.745
[07:49:35.814][HyperV         ][Info   ] Stopping VM MobyLinuxVM...
[07:49:39.673][Moby           ][Info   ] Calibrating delay loop... 
[07:49:39.673][Moby           ][Error  ] Failed to read Moby's logs: The operation has timed out.
[07:49:40.016][HyperV         ][Info   ] Turn Off VM ...
[07:49:40.165][HyperV         ][Info   ] VM MobyLinuxVM killed. Waiting for state to change
[07:49:41.216][HyperV         ][Info   ] Killed VM MobyLinuxVM is off
[07:49:41.217][HyperV         ][Info   ] Destroying Switch DockerNAT...
[07:49:42.876][HyperV         ][Info   ] Removing VM MobyLinuxVM...
[07:49:43.210][HyperV         ][Info   ] Delete VHD C:\Users\Public\Documents\Hyper-V\Virtual Hard Disks\MobyLinuxVM.vhdx
[07:49:43.221][HyperV         ][Debug  ] [stop] took 00:00:07.4960895 to run
[07:49:43.222][OptimizeDisk   ][Info   ] Optimize
[07:49:43.222][PowerShell     ][Info   ] Run script...
[07:49:43.259][Moby           ][Info   ] Stop
[07:49:43.261][ApiProxy       ][Info   ] Killing existing com.docker.proxy with PID 6656
[07:49:43.276][VpnKit         ][Info   ] Killing existing com.docker.slirp with PID 6764
[07:49:43.298][DataKit        ][Info   ] Killing existing com.docker.db with PID 3156
[07:49:43.309][HyperV         ][Info   ] Create
[07:49:43.310][HyperVGuids    ][Info   ] Installing GUIDs...
[07:49:43.310][PowerShell     ][Info   ] Run script with parameters: -Create True -SwitchSubnetAddress 10.0.75.0 -SwitchSubnetMaskSize 24 -CPUs 2 -Memory 2048 -IsoFile C:\Program Files\Docker\Docker\Resources\mobylinux.iso...
[07:49:43.310][PowerMode      ][Info   ] Start
[07:49:43.310][Firewall       ][Info   ] Opening ports for C:\Program Files\Docker\Docker\Resources\com.docker.proxy.exe...
[07:49:43.310][HyperVGuids    ][Info   ] GUIDs installed
[07:49:43.325][HyperV         ][Info   ] Script started at 07:49:43.325
[07:49:43.337][HyperV         ][Info   ] Module loaded at 07:49:43.337
[07:49:43.447][HyperV         ][Info   ] Creating Switch: DockerNAT...
[07:49:47.249][HyperV         ][Info   ] Switch created.
[07:49:48.202][HyperV         ][Info   ] Set IP address on switch
[07:49:48.251][HyperV         ][Info   ] Creating VM MobyLinuxVM...
[07:49:50.636][Firewall       ][Info   ] Ports are opened
[07:49:51.229][HyperV         ][Info   ] Setting CPUs to 2 and Memory to 2048 MB
[07:49:51.802][HyperV         ][Info   ] Creating dynamic VHD: C:\Users\Public\Documents\Hyper-V\Virtual Hard Disks\MobyLinuxVM.vhdx
[07:49:53.136][HyperV         ][Info   ] Attach VHD C:\Users\Public\Documents\Hyper-V\Virtual Hard Disks\MobyLinuxVM.vhdx
[07:49:54.930][HyperV         ][Info   ] Connect Internal Switch DockerNAT
[07:49:55.991][HyperV         ][Info   ] Attach DVD C:\Program Files\Docker\Docker\Resources\mobylinux.iso
[07:49:59.358][HyperV         ][Info   ] VM created.
[07:49:59.359][HyperV         ][Debug  ] [create] took 00:00:16.0500250 to run
[07:49:59.359][PowerShell     ][Info   ] Run script '(Get-VM MobyLinuxVM).Id.Guid'...
[07:49:59.463][DataKit        ][Info   ] Starting C:\Program Files\Docker\Docker\Resources\com.docker.db.exe --url hyperv-connect://98b1c1f2-4ffd-4f43-89e8-684449225e27,\\.\pipe\dockerDataBase --color never
[07:49:59.465][DataKit        ][Info   ] Started
[07:49:59.465][Database       ][Info   ] Generating CA Cert Bundle
[07:49:59.491][Database       ][Info   ] CA Cert Bundle Generated
[07:49:59.536][DataKit        ][Info   ] Starting com.docker.db.exe 0.7.0-34-g6f04213 ...
[07:49:59.553][Database       ][Info   ] Writing com.docker.driver.amd64-linux/net/config=static
[07:49:59.553][Database       ][Info   ] Writing com.docker.driver.amd64-linux/net/address=10.0.75.2
[07:49:59.553][Database       ][Info   ] Writing com.docker.driver.amd64-linux/net/netmask=255.255.255.0
[07:49:59.553][Database       ][Info   ] Writing com.docker.driver.amd64-linux/net/gateway=10.0.75.1
[07:49:59.553][Database       ][Info   ] Writing com.docker.driver.amd64-linux/network=hybrid
[07:49:59.553][Database       ][Info   ] Writing com.docker.driver.amd64-linux/etc/docker/daemon.json={"registry-mirrors":[],"insecure-registries":[]}
[07:49:59.553][Database       ][Info   ] Writing com.docker.driver.amd64-linux/etc/sysctl.conf=
[07:49:59.553][Database       ][Info   ] Writing com.docker.driver.amd64-linux/native/port-forwarding=true
[07:49:59.553][DataKit        ][Info   ] com.docker.db.exe: accepted a new connection on \\.\pipe\dockerDataBase
[07:49:59.627][DataKit        ][Info   ] com.docker.db.exe: accepted a new connection on \\.\pipe\dockerDataBase
[07:50:30.726][Database       ][Error  ] 2016/11/30 07:50:29 Failed to establish 9P session to i/o timeout
2016/11/30 07:50:29 Failed to establish 9P session with the database on \\.\pipe\dockerDataBase: i/o timeout

[07:50:30.726][Database       ][Error  ] Unable to modify database: Unable to write to the database. Exit code: 1. Let's retry in 1s
[07:50:31.731][Database       ][Info   ] Writing com.docker.driver.amd64-linux/net/config=static
[07:50:31.731][Database       ][Info   ] Writing com.docker.driver.amd64-linux/net/address=10.0.75.2
[07:50:31.731][Database       ][Info   ] Writing com.docker.driver.amd64-linux/net/netmask=255.255.255.0
[07:50:31.731][Database       ][Info   ] Writing com.docker.driver.amd64-linux/net/gateway=10.0.75.1
[07:50:31.731][Database       ][Info   ] Writing com.docker.driver.amd64-linux/network=hybrid
[07:50:31.731][Database       ][Info   ] Writing com.docker.driver.amd64-linux/etc/docker/daemon.json={"registry-mirrors":[],"insecure-registries":[]}
[07:50:31.731][Database       ][Info   ] Writing com.docker.driver.amd64-linux/etc/sysctl.conf=
[07:50:31.731][Database       ][Info   ] Writing com.docker.driver.amd64-linux/native/port-forwarding=true
[07:50:31.777][DataKit        ][Info   ] com.docker.db.exe: accepted a new connection on \\.\pipe\dockerDataBase
[07:50:31.779][DataKit        ][Error  ] com.docker.db.exe: Caught End_of_file: closing connection
[07:50:31.779][DataKit        ][Info   ] com.docker.db.exe: Using protocol TwoThousand msize 16384
[07:50:32.010][Database       ][Info   ] Successfully wrote 8 value(s)
[07:50:32.010][Database       ][Info   ] Writing com.docker.driver.amd64-linux/etc/ssl/certs/ca-certificates.crt=C:\WINDOWS\TEMP\tmp4658.tmp
[07:50:32.051][DataKit        ][Info   ] com.docker.db.exe: accepted a new connection on \\.\pipe\dockerDataBase
[07:50:32.051][DataKit        ][Info   ] com.docker.db.exe: Using protocol TwoThousand msize 16384
[07:50:32.124][Database       ][Info   ] Successfully wrote value
[07:50:32.125][Moby           ][Info   ] Connecting...
[07:50:32.125][Moby           ][Error  ] Failed to read Moby's logs: The operation has timed out.
[07:50:32.126][HyperV         ][Info   ] Start
[07:50:32.126][PowerShell     ][Info   ] Run script '$(Find-NetRoute -RemoteIPAddress 8.8.8.8).InterfaceIndex[0]'...
[07:50:32.126][PowerShell     ][Info   ] Run script with parameters: -Start True...
[07:50:32.142][HyperV         ][Info   ] Script started at 07:50:32.142
[07:50:32.152][HyperV         ][Info   ] Module loaded at 07:50:32.152
[07:50:32.156][HyperV         ][Info   ] Starting VM MobyLinuxVM...
[07:50:32.792][Database       ][Info   ] Writing com.docker.driver.amd64-linux/slirp/dns=nameserver 10.2.1.5
nameserver 10.2.1.6
search TRS.STATE.IL.US
[07:50:32.839][DataKit        ][Info   ] com.docker.db.exe: accepted a new connection on \\.\pipe\dockerDataBase
[07:50:32.839][DataKit        ][Info   ] com.docker.db.exe: Using protocol TwoThousand msize 16384
[07:50:33.048][Database       ][Info   ] Successfully wrote 1 value(s)
[07:50:33.079][VpnKit         ][Info   ] Starting C:\Program Files\Docker\Docker\Resources\com.docker.slirp.exe --ethernet hyperv-connect://98b1c1f2-4ffd-4f43-89e8-684449225e27 --port hyperv-connect://98b1c1f2-4ffd-4f43-89e8-684449225e27 --db \\.\pipe\dockerDataBase --debug --diagnostics \\.\pipe\dockerVpnKitDiagnostics
[07:50:33.081][VpnKit         ][Info   ] Started
[07:50:33.300][HyperV         ][Debug  ] [start] took 00:00:01.1743735 to run
[07:50:33.300][VpnKit         ][Info   ] com.docker.slirp.exe: Logging to stdout (stdout:true DEBUG:false)
[07:50:33.300][VpnKit         ][Info   ] com.docker.slirp.exe: Setting handler to ignore all SIGPIPE signals
[07:50:33.300][VpnKit         ][Info   ] com.docker.slirp.exe: vpnkit version %VERSION% with hostnet version  %HOSTNET_PINNED% uwt version 0.0.3 hvsock version 0.11.1 %HVSOCK_PINNED%
[07:50:33.300][VpnKit         ][Info   ] com.docker.slirp.exe: starting port forwarding server on port_control_url:hyperv-connect://98b1c1f2-4ffd-4f43-89e8-684449225e27 vsock_path:
[07:50:33.300][VpnKit         ][Info   ] com.docker.slirp.exe: connecting to 98b1c1f2-4ffd-4f43-89e8-684449225e27:0B95756A-9985-48AD-9470-78E060895BE7
[07:50:33.300][VpnKit         ][Debug  ] com.docker.slirp.exe: Waiting for connections on socket hyperv-connect://98b1c1f2-4ffd-4f43-89e8-684449225e27
[07:50:33.300][VpnKit         ][Info   ] com.docker.slirp.exe: connecting to 98b1c1f2-4ffd-4f43-89e8-684449225e27:30D48B34-7D27-4B0B-AAAF-BBBED334DD59
[07:50:33.300][VpnKit         ][Debug  ] com.docker.slirp.exe: Waiting for connections on socket hyperv-connect://98b1c1f2-4ffd-4f43-89e8-684449225e27
[07:50:33.300][VpnKit         ][Error  ] com.docker.slirp.exe: While watching /etc/resolv.conf: ENOENT
[07:50:33.300][VpnKit         ][Info   ] com.docker.slirp.exe: hosts file has bindings for 
[07:50:33.301][Linux          ][Info   ] Trying to connect to vsud...
[07:50:34.250][Moby           ][Info   ] Connected
[07:50:34.887][Moby           ][Info   ] done.
[07:50:34.887][Moby           ][Info   ] Booting the kernel.
[07:50:34.897][Moby           ][Info   ] Linux version 4.8.10-moby (root@1d811a9194c4) (gcc version 5.3.0 (Alpine 5.3.0) ) #1 SMP Mon Nov 21 22:02:05 UTC 2016
[07:50:34.902][Moby           ][Info   ] Command line: earlyprintk=ttyS0,115200 console=ttyS0,115200 mobyplatform=windows rootdelay=300 noautodetect
[07:50:34.902][Moby           ][Info   ] 
[07:50:34.908][Moby           ][Info   ] Disabled fast string operations
[07:50:34.912][Moby           ][Info   ] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[07:50:34.917][Moby           ][Info   ] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[07:50:34.922][Moby           ][Info   ] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[07:50:34.929][Moby           ][Info   ] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[07:50:34.929][Moby           ][Info   ] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[07:50:34.936][Moby           ][Info   ] x86/fpu: Using 'eager' FPU context switches.
[07:50:34.941][Moby           ][Info   ] e820: BIOS-provided physical RAM map:
[07:50:34.945][Moby           ][Info   ] BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable
[07:50:34.950][Moby           ][Info   ] BIOS-e820: [mem 0x0000000000100000-0x000000007eeedfff] usable
[07:50:34.953][Moby           ][Info   ] BIOS-e820: [mem 0x000000007eeee000-0x000000007eef1fff] ACPI data
[07:50:34.958][Moby           ][Info   ] BIOS-e820: [mem 0x000000007eef2000-0x000000007ef1afff] reserved
[07:50:34.964][Moby           ][Info   ] BIOS-e820: [mem 0x000000007ef1b000-0x000000007ff9afff] usable
[07:50:34.969][Moby           ][Info   ] BIOS-e820: [mem 0x000000007ff9b000-0x000000007fff2fff] reserved
[07:50:34.974][Moby           ][Info   ] BIOS-e820: [mem 0x000000007fff3000-0x000000007fffafff] ACPI data
[07:50:34.978][Moby           ][Info   ] BIOS-e820: [mem 0x000000007fffb000-0x000000007fffefff] ACPI NVS
[07:50:34.983][Moby           ][Info   ] BIOS-e820: [mem 0x000000007ffff000-0x000000007fffffff] usable
[07:50:34.988][Moby           ][Info   ] bootconsole [earlyser0] enabled
[07:50:34.992][Moby           ][Info   ] NX (Execute Disable) protection: active
[07:50:34.998][Moby           ][Info   ] efi: EFI v2.31 by EDK II
[07:50:35.009][Moby           ][Info   ] efi:  ACPI=0x7fffa000  ACPI 2.0=0x7fffa014  SMBIOS=0x7fff2000  PROP=0x4fa13d8  MEMATTR=0x7f525d18 
[07:50:35.012][Moby           ][Info   ] SMBIOS 2.4 present.
[07:50:35.019][Moby           ][Info   ] Hypervisor detected: Microsoft HyperV
[07:50:35.023][Moby           ][Info   ] HyperV: features 0x2e7f, hints 0x2c2c
[07:50:35.027][Moby           ][Info   ] HyperV: LAPIC Timer Frequency: 0x1e8480
[07:50:35.032][Moby           ][Info   ] clocksource: hyperv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x24e6a1710, max_idle_ns: 440795202120 ns
[07:50:35.036][Moby           ][Info   ] tsc: Marking TSC unstable due to running on Hyper-V
[07:50:35.041][Moby           ][Info   ] e820: last_pfn = 0x80000 max_arch_pfn = 0x400000000
[07:50:35.046][Moby           ][Info   ] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WC  UC- WT  
[07:50:35.050][Moby           ][Info   ] RAMDISK: [mem 0x7882c000-0x7b87dfff]
[07:50:35.055][Moby           ][Info   ] ACPI: Early table checksum verification disabled
[07:50:35.061][Moby           ][Info   ] ACPI: RSDP 0x000000007FFFA014 000024 (v02 VRTUAL)
[07:50:35.068][Moby           ][Info   ] ACPI: XSDT 0x000000007FFF90E8 00005C (v01 VRTUAL MICROSFT 00000001 MSFT 00000001)
[07:50:35.081][Moby           ][Info   ] ACPI: FACP 0x000000007FFF7000 0000F4 (v04 VRTUAL MICROSFT 00000001 MSFT 00000001)
[07:50:35.087][Moby           ][Info   ] ACPI: DSDT 0x000000007EEEF000 00256F (v02 MSFTVM DSDT01   00000001 MSFT 04000000)
[07:50:35.094][Moby           ][Info   ] ACPI: FACS 0x000000007FFFE000 000040
[07:50:35.102][Moby           ][Info   ] ACPI: APIC 0x000000007FFF8000 0007D2 (v02 VRTUAL MICROSFT 00000001 MSFT 00000001)
[07:50:35.115][Moby           ][Info   ] ACPI: OEM0 0x000000007FFF6000 000064 (v01 VRTUAL MICROSFT 00000001 MSFT 00000001)
[07:50:35.127][Moby           ][Info   ] ACPI: WAET 0x000000007FFF5000 000028 (v01 VRTUAL MICROSFT 00000001 MSFT 00000001)
[07:50:35.134][Moby           ][Info   ] ACPI: OEM1 0x000000007FFF4000 00009E (v02 MSFTVM UARTS    00000001 MSFT 04000000)
[07:50:35.141][Moby           ][Info   ] ACPI: OEM2 0x000000007FFF3000 0002E2 (v02 MSFTVM VTPM2    00000001 MSFT 04000000)
[07:50:35.149][Moby           ][Info   ] ACPI: SRAT 0x000000007EEEE000 000190 (v02 VRTUAL MICROSFT 00000001 MSFT 00000001)
[07:50:35.173][Moby           ][Info   ] Zone ranges:
[07:50:35.179][Moby           ][Info   ]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[07:50:35.185][Moby           ][Info   ]   DMA32    [mem 0x0000000001000000-0x000000007fffffff]
[07:50:35.190][Moby           ][Info   ]   Normal   empty
[07:50:35.194][Moby           ][Info   ] Movable zone start for each node
[07:50:35.197][Moby           ][Info   ] Early memory node ranges
[07:50:35.202][Moby           ][Info   ]   node   0: [mem 0x0000000000001000-0x000000000009ffff]
[07:50:35.206][Moby           ][Info   ]   node   0: [mem 0x0000000000100000-0x000000007eeedfff]
[07:50:35.211][Moby           ][Info   ]   node   0: [mem 0x000000007ef1b000-0x000000007ff9afff]
[07:50:35.215][Moby           ][Info   ]   node   0: [mem 0x000000007ffff000-0x000000007fffffff]
[07:50:35.220][Moby           ][Info   ] Initmem setup node 0 [mem 0x0000000000001000-0x000000007fffffff]
[07:50:35.228][Moby           ][Info   ] ACPI: PM-Timer IO Port: 0x408
[07:50:35.233][Moby           ][Info   ] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[07:50:35.240][Moby           ][Info   ] IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
[07:50:35.244][Moby           ][Info   ] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[07:50:35.248][Moby           ][Info   ] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[07:50:35.252][Moby           ][Info   ] Using ACPI (MADT) for SMP configuration information
[07:50:35.256][Moby           ][Info   ] smpboot: 240 Processors exceeds NR_CPUS limit of 128
[07:50:35.260][Moby           ][Info   ] smpboot: Allowing 128 CPUs, 126 hotplug CPUs
[07:50:35.264][Moby           ][Info   ] e820: [mem 0x80000000-0xffffffff] available for PCI devices
[07:50:35.269][Moby           ][Info   ] Booting paravirtualized kernel on bare hardware
[07:50:35.274][Moby           ][Info   ] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[07:50:35.279][Moby           ][Info   ] setup_percpu: NR_CPUS:128 nr_cpumask_bits:128 nr_cpu_ids:128 nr_node_ids:1
[07:50:35.328][Moby           ][Info   ] percpu: Embedded 35 pages/cpu @ffff880076800000 s105048 r8192 d30120 u262144
[07:50:35.333][Moby           ][Info   ] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 515799
[07:50:35.338][Moby           ][Info   ] Kernel command line: earlyprintk=ttyS0,115200 console=ttyS0,115200 mobyplatform=windows rootdelay=300 noautodetect
[07:50:35.338][Moby           ][Info   ] 
[07:50:35.344][Moby           ][Info   ] log_buf_len individual max cpu contribution: 4096 bytes
[07:50:35.349][Moby           ][Info   ] log_buf_len total cpu_extra contributions: 520192 bytes
[07:50:35.352][Moby           ][Info   ] log_buf_len min size: 131072 bytes
[07:50:35.356][Moby           ][Info   ] log_buf_len: 1048576 bytes
[07:50:35.360][Moby           ][Info   ] early log buf free: 122776(93%)
[07:50:35.364][Moby           ][Info   ] PID hash table entries: 4096 (order: 3, 32768 bytes)
[07:50:35.369][Moby           ][Info   ] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
[07:50:35.374][Moby           ][Info   ] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
[07:50:35.388][Moby           ][Info   ] Memory: 1944260K/2096184K available (8442K kernel code, 1394K rwdata, 2848K rodata, 1372K init, 588K bss, 151924K reserved, 0K cma-reserved)
[07:50:35.394][Moby           ][Info   ] Hierarchical RCU implementation.
[07:50:35.399][Moby           ][Info   ] 	Build-time adjustment of leaf fanout to 64.
[07:50:35.404][Moby           ][Info   ] Using NULL legacy PIC
[07:50:35.408][Moby           ][Info   ] NR_IRQS:8448 nr_irqs:1448 0
[07:50:35.416][Moby           ][Info   ] Console: colour dummy device 80x25
[07:50:35.419][Moby           ][Info   ] console [ttyS0] enabled
[07:50:35.423][Moby           ][Info   ] console [ttyS0] enabled
[07:50:35.426][Moby           ][Info   ] bootconsole [earlyser0] disabled
[07:50:35.430][Moby           ][Info   ] bootconsole [earlyser0] disabled
[07:50:35.980][Moby           ][Info   ] tsc: Fast TSC calibration failed
[07:50:35.986][Moby           ][Info   ] tsc: Unable to calibrate against PIT
[07:50:35.992][Moby           ][Info   ] tsc: HPET/PMTIMER calibration failed
[07:51:33.638][NamedPipeClient][Error  ] Unable to send Start: Docker hv-sock proxy (vsudd) is not reachable
[07:51:33.634][NamedPipeServer][Error  ] Unable to execute Start: Docker hv-sock proxy (vsudd) is not reachable    at Docker.Backend.ContainerEngine.Linux.ConnectToVsud(TaskCompletionSource`1 vmId)
   at Docker.Backend.ContainerEngine.Linux.DoStart(Settings settings)
   at Docker.Backend.ContainerEngine.Linux.Start(Settings settings)
   at Docker.Core.Pipe.NamedPipeServer.<>c__DisplayClass8_0.<Register>b__0(Object[] parameters)
   at Docker.Core.Pipe.NamedPipeServer.RunAction(String action, Object[] parameters)
[07:51:33.646][SegmentApi     ][Info   ] Usage statistic: actionSettingsChanged
[07:51:33.646][Notifications  ][Info   ] Docker is running
[07:51:33.650][Notifications  ][Error  ] Docker hv-sock proxy (vsudd) is not reachable
[07:51:33.687][CrashReport    ][Info   ] Preparing package to send with the diagnostics
[07:51:33.694][NamedPipeClient][Info   ] Sending GetDebugInfo()...
[07:51:33.696][NamedPipeClient][Info   ] Sending DownloadVmLogs()...
[07:51:34.641][NamedPipeServer][Info   ] GetDebugInfo()
[07:51:34.648][NamedPipeServer][Info   ] DownloadVmLogs()
[07:51:34.654][PowerShell     ][Info   ] Run script '(Get-VM MobyLinuxVM).Id.Guid'...
[07:51:38.032][NamedPipeClient][Info   ] Received response for DownloadVmLogs
[07:51:38.032][HyperV         ][Warning] Unable to download logs: Connection failed
[07:51:38.032][NamedPipeServer][Info   ] DownloadVmLogs done in 00:00:03.3845970.
[07:52:05.235][NamedPipeServer][Info   ] GetDebugInfo done in 00:00:30.5936622.
[07:52:05.250][NamedPipeClient][Info   ] Received response for GetDebugInfo
[07:52:05.322][CrashReport    ][Info   ] Uploading diagnostics...
[07:52:05.960][CrashReport    ][Info   ] Diagnostics successfully uploaded
[07:52:06.614][GUI            ][Info   ] Sending Bugsnag report 48b96fd1-d886-4981-87d7-14328c383b38...
[07:52:08.945][GUI            ][Info   ] Bugsnag report 48b96fd1-d886-4981-87d7-14328c383b38 sent
[07:54:11.311][ErrorReportWindow][Info   ] Open logs

@Ncage1974 Ncage1974 changed the title Another Failure on running linux containers on windows 10 (beta 31) Nov 30, 2016

@dgageot

This comment has been minimized.

Copy link

dgageot commented Nov 30, 2016

@tverboon

This comment has been minimized.

Copy link

tverboon commented Nov 30, 2016

Same issue for me. I'm trying to run Docker in a Windows 10 VM in Parallels Desktop Pro.

@rn

This comment has been minimized.

Copy link
Contributor

rn commented Nov 30, 2016

@Ncage1974 @tverboon we can't really support running Windows 10 in a VM and then run Docker for Windows on top as this requires nested virtualisation. Getting nested virtualisation right is very hard and we can't control nor check how you configured the VM nor what else is running on the physical host. Therefore I'm closing this issue.

Having said that, we do use nested virt on top of ESXi for some of our testing, but also see intermittent failures due to it, which is another reason we can't support this.

@rn rn closed this Nov 30, 2016

@tverboon

This comment has been minimized.

Copy link

tverboon commented Dec 1, 2016

@rneugeba Thank you for clarifying and I understand that's not worth the effort if it's impossible to get it totally right in all situations and environments.

That said, maybe you should consider removing or changing this blogpost: https://blog.docker.com/2016/04/containers-and-vms-together/

And I guess it could be more clear in: https://docs.docker.com/docker-for-windows/#what-to-know-before-you-install

This information was making me hopeful and I bought the Pro license from Parallels to have virtualization enabled through nested virtualization.

@friism

This comment has been minimized.

Copy link

friism commented Dec 1, 2016

cc @londoncalling it might be worth adding a callout to that "what to know before you install" docs section that nested virtualization scenarios may work, but come with no guarantees.

@rn

This comment has been minimized.

Copy link
Contributor

rn commented Dec 1, 2016

@tverboon thanks for the suggestion of improving the docs. As for the blog post it describes a different scenario.It uses a hypervisor to run different OSes which then then run native docker engines. In Docker for Windows you can run either Linux containers (which run in a Linux VM) or Hyper-V windows container (which also run in a VM), ie you need to be able to run a hypervisor on top of a hypervisor and this scenario is requires something called nested virtualisation.

You seem to be running a Mac. If you want to work with Linux containers, Docker for Mac is a better approach. If you want to work with Windows containers on the Mac, then, for now i would suggest to install Windows 2016 Server and use the native Windows Container feature there. Unlike Hyper-V containers, the native Windows containers on Windows Server do not require nested virtualisation. See https://msdn.microsoft.com/en-us/virtualization/windowscontainers/about/index?f=255&MSPPError=-2147217396 for an explanation of the different container types on Windows.

@tverboon

This comment has been minimized.

Copy link

tverboon commented Dec 1, 2016

@rneugeba I see, thanks.

It is our build server that we need to run on Windows, because code coverage with OpenCover doesn't work with .Net Core on the Mac and now we have a Docker build task. Maybe I could connect to the Docker Host running on the Mac and perform the docker build en docker push(?), but I guess I will configure a Mac mini and boot Windows native with Bootcamp.

@londoncalling

This comment has been minimized.

Copy link

londoncalling commented Dec 3, 2016

@friism , docs updated per PR 780 on docker.github.io.

@tylkomat

This comment has been minimized.

Copy link

tylkomat commented Dec 6, 2016

I have the same issue while not running inside a VM. My ID is: EF5B5C5D-8BE2-400B-B341-39C486F7F6C0/2016-12-06_19-28-48

@yohanb

This comment has been minimized.

Copy link

yohanb commented Dec 7, 2016

Same for me.
When switching from Windows to Linux containers, Docker fails with the following message:
image

  • Docker version: 1.13.0-rc3-beta32 (9266)
  • Windows version: Windows 10 Enterprise, 1607
@tverboon

This comment has been minimized.

Copy link

tverboon commented Jan 12, 2017

@rneugeba Not sure if something has changed with Docker regarding this. But I am now running Parallels on my Late 2015 iMac instead of late 2009. The big difference of course is that my late 2015 iMac has the required hypervisor Intel goodies Intel VT-x feature set that includes Extended Page Tables (EPT)

And docker is running on virtualized Windows in Parallels. Great!

@kutenai

This comment has been minimized.

Copy link

kutenai commented Jan 26, 2017

It is unfortunate that this issue is closed. I have a very specific reason to use Docker under Parallels with windows on my Mac.. I'm supporting others that "do windows", so for me, it's a process development and validation need.

Parallels Pro purportedly support this, so not sure why it's failing. I've spent 2 last two days on and off updating, validating and basically doing everything I can think of to make this work, but it won't cooperate.

@friism

This comment has been minimized.

Copy link

friism commented Jan 26, 2017

@kutenai can you create and run a normal Linux Hyper-V VM inside your Windows parallels VM on your Mac?

@kutenai

This comment has been minimized.

Copy link

kutenai commented Jan 26, 2017

I've never tried. I'll have to google how to do that, but I'll give it a shot.

@kutenai

This comment has been minimized.

Copy link

kutenai commented Jan 26, 2017

Yes, I am able to boot a Hyper-V with an iso image, and it appears to be working as expected.

@tverboon

This comment has been minimized.

Copy link

tverboon commented Jan 26, 2017

@kutenai Which Mac are you running on? Could you run this command in Terminal and see if it returns 1:
sysctl kern.hv_support

On my Late 2009 iMac it didn't work with Parallels 11 or Parallels 12 Pro, my Late 2015 iMac with Parallels 11 is working. I figured the big difference is the Intel processor.

@kutenai

This comment has been minimized.

Copy link

kutenai commented Jan 26, 2017

@tverboon

This comment has been minimized.

Copy link

tverboon commented Jan 26, 2017

Strange thing is that I don't have the Pro edition of Parallels 11. According to Parallels, it should not work. I bought 12 Pro for the old iMac, hoping it would work, but it didn't. I'm running Windows 10 Pro on Parallels 11 on MacOS Sierra.

What is your exact configuration?

@kutenai

This comment has been minimized.

Copy link

kutenai commented Jan 26, 2017

@tverboon

This comment has been minimized.

Copy link

tverboon commented Jan 27, 2017

Well we have the same setup for the most part. The most obvious differences are the underlying hardware and Parallels Desktop version. You could try to run a backup of your VM image on parallels desktop 11: http://www.parallels.com/directdownload/pd11/

Other than that I am out of ideas, because it's either something in Docker or Parallels or both.

@rn

This comment has been minimized.

Copy link
Contributor

rn commented Jan 27, 2017

Docker for Windows should be able to run inside Windows 10 VM running on top of Parallels and or VMware on a Mac provided it the VM is properly configured, and even then it is possible that sometimes issues occur due to the way Parallels and VMware virtualise the Hardware. FWIW, I run Docker for Windows in a Windows VM on VMware Fusion on my Mac and we use Windows VMs on ESXi for testings. And in these environments we do see intermittent failures which are clearly attributable to the fact that we are using nested virtualisation. For these reasons we can't really support this configuration and it's really YMMV. It works for some and it doesn't work for others.

As a general rule, though:

  • Make sure VMWare/Parallels has nested virtualisation support enabled. VMware Fusion has an Advanced Option under the CPU & Memory setting menu, which must be enabled. I don't have Parallels, but from the manual it seems that there is an option in the CPU & Memory setting to enable Nested Virtualisation as well.
  • Configure your VM with at least 2 CPUs and sufficient memory. Mine is configured with 6GB.
  • Make sure your system is more or less idle.
  • Make sure your Windows is up to date. There have been several issues, at least with some insider builds.
  • The processor you have may also be relevant. For example Westmere based Mac Pros have some additional Hardware virtualisation features over Nehalem based Mac Pros and so do newer generations of Intel processors.

Typical failures we see with nested virtualisation:

  • Slow boot time of the Linux VM. If you look in the logs, you'll see some entries prefixed with Moby. On real hardware, it takes 5-10 seconds to boot the Linux VM. Roughly the time between the Connected log entry and the * Starting Docker ... [ ok ] log entry. If you boot the Linux VM inside a Window VM, this may take considerably longer. We have a timeout of 60s or so. If the VM hasn't started by that time we retry. If the retry fails we print an error...You may be able to work around this by providing more resources to the Windows VM.
  • Sometimes the VM fails to boot when Linux tries to calibrate the TSC (Times stamp counter). This process is quite timing sensitive and may fail when executed inside a VM which itself runs inside a VM.

CPU utilisation is also very likely to be higher...

@kutenai

This comment has been minimized.

Copy link

kutenai commented Jan 27, 2017

I have verified all of the items you mention above. I have 8GB and 4 CPU's assigned to the VM, and then 2 CPU's with 2GB assigned to Docker. I have nested virt. enabled, windows is up-to-date, etc.. I also have Westmere processors..

But, as you said, YMMV. I think my next step is to install a fresh copy of Windows 10.. since the one I'm using has a lot of cruft. Windows always did have a tendency to get crufty rather fast.. one of the (many) reasons to prefer OSX.

Thanks for all of the advice and help. I can't thank you all enough, I very much appreciate your time in offering advice and support.

@londoncalling

This comment has been minimized.

Copy link

londoncalling commented Jan 27, 2017

@rneugeba @friism I could add some of this info to the docs, with appropriate caveats, under an FAQ or Troubleshooting somewhere, as a link off of the "not supported" line, if you like. Let me know if this would lead to more or less problems/overhead.

@rn

This comment has been minimized.

Copy link
Contributor

rn commented Jan 28, 2017

@londoncalling yeah, that would be useful. Then I can just link to it if similar issues arise.
thanks

@londoncalling

This comment has been minimized.

Copy link

londoncalling commented Jan 29, 2017

@rneugeba cool, I will do it!

@BretFisher

This comment has been minimized.

Copy link

BretFisher commented Jan 29, 2017

Some more info in my testing. I have a similar setup and same error... Windows 10 Enterprise running on Parallels for Mac 12 Pro. macOS 12 on latest MacBook Pro 13".

I can start Hyper-V and create a Linux VM fine and run it fine. The Moby VM, when the Docker agent starts it, uses 20-25% CPU and I never see it boot in the VM Console, Hyper-V just says "Connecting".

Docker for Mac used to work, many months ago. This also had the same result on my older 2012 MacBook Pro.

I wonder if this isn't a Docker thing specifically, but something with the Moby Alpine Kernel. Docker errors out because it can't connect to the VM, but the VM never boots. Hyper-V can't even get a Console connection to it.

When trying to boot the latest Alpine Linux, it won't boot in this setup either unless I choose the Xen version of the iso, as documented by many here as an ongoing issue with Hyper-V https://bugs.alpinelinux.org/issues/1021

If I take the docker-provided mobylinux.iso and create a new VM with the exact settings (gen 2, disable secure boot, 2gb of ram, no dynamic ram) it exhibits the exact same issue, it starts to boot, never boots, and sits at 20% CPU.

Any chance they are related?

Any way to determine if the mobylinux.iso is same Kernel/Driver setup as one of the stock provided by Alpine Downloads?

@rn

This comment has been minimized.

Copy link
Contributor

rn commented Jan 29, 2017

@BretFisher the MobyLinux VM is configured to not use the console, but the serial console, which the Docker for Windows application is monitoring (and the output should be in the logs). So you won't see anything in the Hyper-V manager console. You also can't just boot the ISO because the Linux inside is configured to contact some services on the host during boot. So, if the app is not running the services are not there and therefore the Linux VM boot does not complete.

the mobylinux.iso does not use the same kernel as alpine linux. we use a recent 4.9.x kernel. I keep a patched kernel at https://github.com/rneugeba/linux-stable

@SetTrend

This comment has been minimized.

Copy link

SetTrend commented Jun 22, 2018

I'm working with VMware Workstation here, and I'm facing the same situation:

docker error



My Windows 10x64 VM is properly configured for virtualization, according to this Microsoft Wiki article:

http://social.technet.microsoft.com/wiki/contents/articles/22283.how-to-install-hyper-v-on-vmware-workstation-10.aspx

I'm working as a consultant, servicing different enterprises, each with their particular software setup. For professional use it's inevitable to utilize virtual machines to have a clean and safe development environment for each project/customer.

Is this a matter of joining heads with VMware to get this problem solved? Or this is issue pertinent to Docker for Windows alone?

@BretFisher

This comment has been minimized.

Copy link

BretFisher commented Jun 24, 2018

@SetTrend I don't see how your issue is related to this GH Issue. You likely have a problem with Hyper-V running properly on your Windows 10 Pro machine. You should verify you can boot a working OS in Hyper-V before you start troubleshooting Docker.

@SetTrend

This comment has been minimized.

Copy link

SetTrend commented Jun 26, 2018

@BretFisher, why in detail do you feel that my Hyper-V installation isn't running properly when I'm reporting a Docker for Windows issue here?

As you can see from the following animation, Hyper-V is running flawlessly from within my VMware virtual machine:

hyper-v runs flawlessly in vmware vm

@rn

This comment has been minimized.

Copy link
Contributor

rn commented Jun 26, 2018

@SetTrend we had a lot of issues with nested virt. Our CI used to run very carefully configured and provisioned Windows VMs in order to test Docker for Windows. We had a lot of transient failures related to nested virt. Some of them are related to the fact that sometimes the Linux VM is stalled because of other activity on the physical host over which we have no control. We also have no visibility on how your windows VM is configured etc.

Basically this means we can't really support this type of configuration. If it works, great...but if it fails we can't really diagnose it properly.

Incidentally the error your are seeing may be related to #1995. We have a timeout to check if the VM is up and the error may happen if the timeout is exceeded (which happens often in nested virt scenarios). You could try the latest edge release where we have changed the timeout handling a little and it may help...

@SetTrend

This comment has been minimized.

Copy link

SetTrend commented Jun 27, 2018

@rn

Thank you for your reply.

Some of them are related to the fact that sometimes the Linux VM is stalled because of other activity on the physical host over which we have no control. We also have no visibility on how your windows VM is configured etc.

That's in fact the basic nature of virtualization. Hypervisors define reality for the guest OS. There is no such thing like a punch-through up the hierarchy of hypervisors 'til you reach the real hardware.

For example, if a hypervisor decides that time elapses in double time in the guest, then, for the guest, time elapses in double time - without it knowing or recognizing.

Put in other words: A VM's reality is defined by the hypervisor. Any information you don't get from the hypervisor just isn't available. There is no fourth dimension or a fifth wall - not for any virtualized operating system. Einstein described it very well in his theory of relativity. If the hypervisor pauses a VM - time stands still in the VM. There is no reason for the VM to doubt that a time lapse didn't truely happen.

We have a timeout to check if the VM is up and the error may happen if the timeout is exceeded

Given my above reasoning, that strategy seems inappropriate for checking whether a VM is up and running. Probably the time base you compare to should be taken from an alternative, VM robust source (e.g. (guest) OS ticks). So I propose you think of a new, safe alternative to check whether a VM is up and running.

May I ask: How mission critical is this check anyway? Would you want to implement an expert option, a Boolean, in Docker for Windows settings, allowing the user to disable/bypass this check?

@huaizhuang700

This comment has been minimized.

Copy link

huaizhuang700 commented Aug 24, 2018

hello

@kstaniek

This comment has been minimized.

Copy link

kstaniek commented Nov 27, 2018

+1

@gillesdouaire

This comment has been minimized.

Copy link

gillesdouaire commented Dec 19, 2018

Same here. I am able to run Linux VMs in HyperV in the Cloud (e.g. nested virtualization), but when it comes to running Docker for Windows, it fails on MobyLinux checks. It's very opaque, and difficult to troubleshoot. Again, Windows users are second class citizens.

@BretFisher

This comment has been minimized.

Copy link

BretFisher commented Dec 21, 2018

@gillesdouaire I don't think you're being fair here. Nested virtualization is hard, and this issue is common on macOS as well. Some setups work, some don't. There are no favorites, just hard problems that everyone involved would love to fix if they had unlimited time and budget.

What I can say works for me in December 2018 with 18.09 release using nested virtualization:

  • Latest Windows 10 VM in Azure, runs Docker for Windows fine with Linux Containers.
  • On my home lab, with host Windows Server 2019, Hyper-V Server enabled, and Windows 10 Pro 1803 as the guest, then Docker for Windows works there as well for Linux Containers. This setup did not work earlier this year with Windows Server 2016 as host. It started working when I did a OS upgrade to 2019.

What does NOT work for me using nested virtualization:

  • macOS as host using Parallels or VirtualBox to run Windows 10
  • Windows Server 2016 as host running Hyper-V... at some point in the last year that broke.
@SetTrend

This comment has been minimized.

Copy link

SetTrend commented Dec 21, 2018

@ DfW contributors/maintainers:

Why don't you use WSL for Docker for Windows in favour of the proprietary MobyLinuxVM?

The WSL Linux VMs run flawlessly, even in nested virtual environments.

@gillesdouaire

This comment has been minimized.

Copy link

gillesdouaire commented Dec 21, 2018

@BretFisher What I am saying is the issue is not with the nested virtualization alone. The issue is also with Docker interacting with the nested virtualization, and Docker team simply refuses to acknowledge that. Some people report timeouts during the MobyLinux VM setup in the nested virtualization context, so why not make that timeout adjustable? The Linuxkit yaml file used to build MobyLinux ISO refers to docker images that are set as private. What isn't that open to help people adjust the kernel version, etc? While Docker may still decide to refuse to officially support it, they could at least make it easier for their customer base to explore and solve it. Who knows, someone out there may finally find a more robust setup... That's why I am not happy with Docker on that topic.

@BretFisher

This comment has been minimized.

Copy link

BretFisher commented Dec 31, 2018

Sorry, but WSL is not a VM. WSL and the LinuxKit VM for Docker are not similar technologies. WSL does limited system call translations from Linux binaries to the Windows Kernel. It isn’t a complete replacement for the Linux Kernel and not recommend uses outside its scope: https://docs.microsoft.com/en-us/windows/wsl/faq#can-i-run-all-linux-apps-in-wsl

@SetTrend

This comment has been minimized.

Copy link

SetTrend commented Dec 31, 2018

Yes, you're absolutely right.

But ...

  • WSL is basically a minimal Linux VM (created by Canonical)
  • DfW doesn't require a GUI in the Linux VM. Docker itself doesn't provide any GUI for containers.
  • If DfW would be using WSL, DfW could basically issue any Bash command there is and use Linux Containers from within WSL. If there'd be any issues running WSL by DfW, then I'm confident the Microsoft team will be very willing to fill the gaps required to support DfW.

Currently, from my point of view, there is no real reason for utilizing WSL. The benefits compared to installing a Linux VM reach zero. It's plain easier and more versatile to simply install/clone a Linux VM and enable file sharing. So WSL lacks a reason for existing.

Joining DfW and WSL would support both projects: WSL would receive a bold reason for existing, and DfW would finally run Linux containers without flaw. And both would take a great leap towards integrating the Windows OS a bit more into current web development.

My 2 cents. I might be wrong, though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment