New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No internet access with native Mac Kubernetes (docker-for-desktop) #736

Closed
soujiro32167 opened this Issue Aug 14, 2018 · 6 comments

Comments

5 participants
@soujiro32167
Copy link

soujiro32167 commented Aug 14, 2018

Description

After launching telepresence (without any arguments), domains from Kubernetes are accessible to Chrome, but nothing outside of Kubernetes.

Otherwise, containers can see the internet:

➜  ~ k exec debugger-9bc78cc87-ngmfq -- ping github.com
PING github.com (192.30.253.113): 56 data bytes
64 bytes from 192.30.253.113: seq=42836 ttl=37 time=28.592 ms
64 bytes from 192.30.253.113: seq=1 ttl=37 time=29.497 ms

With gcloud (GKE) and telepresence, both Kubernetes and all other domains work fine

Details

  • Docker version
Client:
 Version:           18.06.0-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        0ffa825
 Built:             Wed Jul 18 19:05:26 2018
 OS/Arch:           darwin/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.06.0-ce
  API version:      1.38 (minimum version 1.12)
  Go version:       go1.10.3
  Git commit:       0ffa825
  Built:            Wed Jul 18 19:13:46 2018
  OS/Arch:          linux/amd64
  Experimental:     false
 Kubernetes:
  Version:          v1.10.3
  StackAPI:         v1beta2
  • Telepresence version: 0.90
  • K8s version: 1.10.3

From telepresence.log:

92.8  18 | 2018-08-14T20:54:25+0000 [stdout#info] Failed to lookup b'safebrowsing.googleapis.com' due to [Failure instance: Traceback (failure with no frames): <class 'twisted.names.error.DomainError'>: [Errno -2] Name does not resolve
  92.8  18 | 2018-08-14T20:54:25+0000 [stdout#info] ], falling back to b'safebrowsing.googleapis.com.wework.com'
  92.9  18 | 2018-08-14T20:54:25+0000 [-] DNSDatagramProtocol starting on 20818
  92.9  18 | 2018-08-14T20:54:25+0000 [-] Starting protocol <twisted.names.dns.DNSDatagramProtocol object at 0x7fb83282fb38>
  92.9  18 | 2018-08-14T20:54:25+0000 [-] (UDP Port 7796 Closed)
  92.9  18 | 2018-08-14T20:54:25+0000 [-] Stopping protocol <twisted.names.dns.DNSDatagramProtocol object at 0x7fb8327ef208>
  93.2  18 | 2018-08-14T20:54:25+0000 [stdout#info] getaddrinfo error: [Errno -3] Try again
  93.2  18 | 2018-08-14T20:54:25+0000 [stdout#info] getaddrinfo error: [Errno -2] Name does not resolve
  95.9  18 | 2018-08-14T20:54:28+0000 [-] Unhandled Error
  95.9  18 | 	Traceback (most recent call last):
  95.9  18 | 	Failure: twisted.internet.defer.TimeoutError: [Query('assets-cdn.github.com.wework.com', 1, 1)]
  95.9  18 |
  96.0  18 | 2018-08-14T20:54:28+0000 [-] Unhandled Error
  96.0  18 | 	Traceback (most recent call last):
  96.0  18 | 	Failure: twisted.internet.defer.TimeoutError: [Query('assets-cdn.github.com.wework.com', 1, 1)]
  96.0  18 |
  96.0  18 | 2018-08-14T20:54:28+0000 [-] Unhandled Error
  96.0  18 | 	Traceback (most recent call last):
  96.0  18 | 	Failure: twisted.internet.defer.TimeoutError: [Query('assets-cdn.github.com.wework.com', 1, 1)]
  96.0  18 |
  96.0  18 | 2018-08-14T20:54:28+0000 [-] Unhandled Error
  96.0  18 | 	Traceback (most recent call last):
  96.0  18 | 	Failure: twisted.internet.defer.TimeoutError: [Query('assets-cdn.github.com.wework.com', 1, 1)]
  96.0  18 |
  96.0  18 | 2018-08-14T20:54:28+0000 [-] Unhandled Error
  96.0  18 | 	Traceback (most recent call last):
  96.0  18 | 	Failure: twisted.internet.defer.TimeoutError: [Query('assets-cdn.github.com.wework.com', 1, 1)]
  96.0  18 |
  96.0  18 | 2018-08-14T20:54:28+0000 [-] Unhandled Error
  96.0  18 | 	Traceback (most recent call last):
  96.0  18 | 	Failure: twisted.internet.defer.TimeoutError: [Query('assets-cdn.github.com.wework.com', 1, 1)]
  96.0  18 |
  96.0  18 | 2018-08-14T20:54:28+0000 [-] Unhandled Error
  96.0  18 | 	Traceback (most recent call last):
  96.0  18 | 	Failure: twisted.internet.defer.TimeoutError: [Query('assets-cdn.github.com.wework.com', 1, 1)]
  96.0  18 |
  96.0  18 | 2018-08-14T20:54:28+0000 [-] Unhandled Error
  96.0  18 | 	Traceback (most recent call last):
  96.0  18 | 	Failure: twisted.internet.defer.TimeoutError: [Query('assets-cdn.github.com.wework.com', 1, 1)]
  96.0  18 |
  96.0  18 | 2018-08-14T20:54:28+0000 [-] (UDP Port 61835 Closed)
  96.0  18 | 2018-08-14T20:54:28+0000 [-] Stopping protocol <twisted.names.dns.DNSDatagramProtocol object at 0x7fb832841860>
  97.7  25 | c : DNS request from ('10.65.108.0', 24811) to None: 45 bytes
  97.7  18 | 2018-08-14T20:54:29+0000 [stdout#info] A query: b'safebrowsing.googleapis.com'
  97.8  18 | 2018-08-14T20:54:30+0000 [stdout#info] getaddrinfo error: [Errno -2] Name does not resolve
  97.8  25 | c : DNS request from ('10.65.108.0', 55676) to None: 56 bytes
  97.8  18 | 2018-08-14T20:54:30+0000 [stdout#info] Updated query of type 1 from b'safebrowsing.googleapis.com.wework.com' to b'safebrowsing.googleapis.com'
  97.8  18 | 2018-08-14T20:54:30+0000 [stdout#info] A query: b'safebrowsing.googleapis.com'
  97.8  18 | 2018-08-14T20:54:30+0000 [stdout#info] getaddrinfo error: [Errno -2] Name does not resolve
  97.8  18 | 2018-08-14T20:54:30+0000 [stdout#info] Failed to lookup b'safebrowsing.googleapis.com' due to [Failure instance: Traceback (failure with no frames): <class 'twisted.names.error.DomainError'>: [Errno -2] Name does not resolve
  97.9  18 | 2018-08-14T20:54:30+0000 [stdout#info] ], falling back to b'safebrowsing.googleapis.com.wework.com'
 100.8  25 | c : DNS request from ('10.65.108.0', 37037) to None: 27 bytes
 100.8  18 | 2018-08-14T20:54:33+0000 [stdout#info] A query: b'ide.c9.io'
 100.8  18 | 2018-08-14T20:54:33+0000 [stdout#info] getaddrinfo error: [Errno -2] Name does not resolve
 100.8  25 | c : DNS request from ('10.65.108.0', 55315) to None: 38 bytes
 100.8  18 | 2018-08-14T20:54:33+0000 [stdout#info] Updated query of type 1 from b'ide.c9.io.wework.com' to b'ide.c9.io'
 100.8  18 | 2018-08-14T20:54:33+0000 [stdout#info] A query: b'ide.c9.io'
 100.8  18 | 2018-08-14T20:54:33+0000 [stdout#info] getaddrinfo error: [Errno -2] Name does not resolve
 100.8  18 | 2018-08-14T20:54:33+0000 [stdout#info] Failed to lookup b'ide.c9.io' due to [Failure instance: Traceback (failure with no frames): <class 'twisted.names.error.DomainError'>: [Errno -2] Name does not resolve
 100.8  18 | 2018-08-14T20:54:33+0000 [stdout#info] ], falling back to b'ide.c9.io.wework.com'
 102.8  25 | c : DNS request from ('10.65.108.0', 44676) to None: 56 bytes
 102.8  18 | 2018-08-14T20:54:35+0000 [stdout#info] Updated query of type 1 from b'safebrowsing.googleapis.com.wework.com' to b'safebrowsing.googleapis.com'
 102.8  18 | 2018-08-14T20:54:35+0000 [stdout#info] A query: b'safebrowsing.googleapis.com'
 102.9  18 | 2018-08-14T20:54:35+0000 [stdout#info] getaddrinfo error: [Errno -2] Name does not resolve
 102.9  18 | 2018-08-14T20:54:35+0000 [stdout#info] Failed to lookup b'safebrowsing.googleapis.com' due to [Failure instance: Traceback (failure with no frames): <class 'twisted.names.error.DomainError'>: [Errno -2] Name does not resolve
 102.9  18 | 2018-08-14T20:54:35+0000 [stdout#info] ], falling back to b'safebrowsing.googleapis.com.wework.com'
@ark3

This comment has been minimized.

Copy link
Contributor

ark3 commented Aug 14, 2018

Thanks for the issue. I believe this problem is due to a DNS loop. We need to update check_if_in_local_vm to catch this case.

@ark3 ark3 added the bug label Aug 14, 2018

@ark3 ark3 added this to To do in Tel Tracker via automation Aug 14, 2018

@soujiro32167

This comment has been minimized.

Copy link
Author

soujiro32167 commented Aug 15, 2018

Thanks for the super quick reply!

@thedodd

This comment has been minimized.

Copy link

thedodd commented Sep 13, 2018

I'm running into the same issue. Only when running a local process, not when using --docker-run.

@thedodd

This comment has been minimized.

Copy link

thedodd commented Sep 13, 2018

It also seems as though my process running under --docker-run can not communicate with other services. The telepresence log is showing that an A record is resolved with a specific IP, sends it back, but my process is not able to connect to it.

Could be related. Also, it is an HTTP/2 gRPC connection which is failing. Figured I would add that in as context.

@vemv

This comment has been minimized.

Copy link

vemv commented Nov 5, 2018

This bug is obviously pretty bad - you'll agree what it's hard to develop (or work at all: think Slack, Email etc) without an internet connection.

Could I please have an approximate ETA for a fix? Not sure if it'll come reasonably soon or if contrariwise you're overwhelmed with issues.

Alternatively, is there a workaround? (which hopefully still allows one to use the VPN method, so IDEs/debuggers work)

@rohansingh

This comment has been minimized.

Copy link
Contributor

rohansingh commented Jan 24, 2019

The fix for this is actually very simple, I'll send a PR in a minute. In the meantime, you can temporarily work around this by running kubectl config rename-context docker-for-desktop minikube.

That will rename the context to minikube, which is handled appropriately by _check_if_in_local_vm. YMMV on how this may affect anything else you're using, though.

rohansingh added a commit to rohansingh/telepresence that referenced this issue Jan 24, 2019

Fix outbound network access with Docker Desktop
Docker Desktop has the same issue as minikube/minishift, so we need to
detect and handle it the same way.

Fixes telepresenceio#736.

rohansingh added a commit to rohansingh/telepresence that referenced this issue Jan 24, 2019

Fix outbound network access with Docker Desktop
Docker Desktop has the same issue as minikube/minishift, so we need to
detect and handle it the same way.

Fixes telepresenceio#736.

rohansingh added a commit to rohansingh/telepresence that referenced this issue Jan 24, 2019

Fix outbound network access with Docker Desktop
Docker Desktop has the same issue as minikube/minishift, so we need to
detect and handle it the same way.

Fixes telepresenceio#736.

@ark3 ark3 closed this in #905 Jan 24, 2019

Tel Tracker automation moved this from To do to Done Jan 24, 2019

ark3 added a commit that referenced this issue Jan 24, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment