New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TCP broken pipe docker for mac #2442

Closed
alexschultz opened this Issue Jan 16, 2018 · 9 comments

Comments

Projects
None yet
6 participants
@alexschultz

alexschultz commented Jan 16, 2018

Expected behavior

We have a connection pool used for connecting to a database through a VPN and in code the connections are set to refresh every 5 minutes.

Actual behavior

The TCP connections are breaking randomly in the container. I see multiple TCP broken Pipe errors. I am able to reestablish connections to the database from within the container, but I cannot figure out why the connections are breaking like this. We have tested this in older version of docker4mac and it does not happen.

Information

EDAE6940-D0E4-4F11-81AE-77CF4974281E

Docker for Mac: version: 17.12.0-ce-mac47 (72b93a017350990850ddc37cd341bd16fce3e911)
macOS: version 10.13.2 (build: 17C88)
logs: /tmp/EDAE6940-D0E4-4F11-81AE-77CF4974281E/20180116-130546.tar.gz
[OK] db.git
[OK] vmnetd
[OK] dns
[OK] driver.amd64-linux
[OK] virtualization VT-X
[OK] app
[OK] moby
[OK] system
[OK] moby-syslog
[OK] kubernetes
[OK] env
[OK] virtualization kern.hv_support
[OK] slirp
[OK] osxfs
[OK] moby-console
[OK] logs
[OK] docker-cli
[OK] menubar
[OK] disk

@djs55

This comment has been minimized.

Contributor

djs55 commented Jan 18, 2018

Thanks for your report.

The networking service vpnkit periodically checks for idle TCP connections and times them out. It turns out that while the configuration system was revamped in 17.12 the threshold was accidentally dropped to 30s, checked every 30s so the effect would kick in after 30s-60s -- I suspect this is the cause of your problem.

I've got yet another experimental build: https://download-stage.docker.com/mac/bysha1/d459d63b84c1968dcacd7597133d5ee9c3c6509f/Docker.dmg this one contains the HTTP proxy fixes plus uses the higher timeout value. Could you let me know if it helps?

@alexschultz

This comment has been minimized.

alexschultz commented Jan 18, 2018

@djs55

This comment has been minimized.

Contributor

djs55 commented Jan 18, 2018

@coredumperror

This comment has been minimized.

coredumperror commented Jan 19, 2018

OMG this is what was causing my remote debugger sessions to die! I just installed the new edge build, checked the changelog and saw this issue mentioned, and then eagerly checked to see if my debug sessions were still dying after about 30 seconds. And they're not! It took 5 minutes of idling to die this time.

I do wonder, though: Is it possible for the user to configure that 5-minute timeout to be even longer? 300 seconds should be plenty most of the time, but in that rare case where its not, it'll be super frustrating to have my debug session suddenly die. I'd love to just make vpnkit not kill idle connections at all, if that's possible.

@djs55

This comment has been minimized.

Contributor

djs55 commented Jan 19, 2018

@coredumperror thanks for the confirmation. I agree, I'd like to remove this timeout altogether. The only case it's needed for currently if when a container is torn down and the IP released -- the TCP sessions take a long time to die in that case
I've got some patches which try to enable TCP keepalives from the Mac to the VM. I'm hoping that this will allow us to expire individual connections which are genuinely dead, while keeping alive ones open.

@coredumperror

This comment has been minimized.

coredumperror commented Jan 19, 2018

That'd be great! I would definitely like to not have to worry at all about leaving a debug session idle.

@jdmold

This comment has been minimized.

jdmold commented Jan 22, 2018

We were experiencing this same problem. Updating to mac49 seems to have fixed it.

@docker-for-desktop-robot

This comment has been minimized.

Collaborator

docker-for-desktop-robot commented May 9, 2018

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

@djs55

This comment has been minimized.

Contributor

djs55 commented May 9, 2018

I think this issue is now fixed so I'll close it. Feel free to open a fresh issue if you encounter further problems!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment