github.com not reachable from alpine image #155
Comments
|
Duplicate of #151 but given the significance maybe makes sense to report twice 🤣 |
|
I've got a similar problem with alpine:3.13 which doesn't exist in alpine:3.12 The above snippet is the most basic illustration of my issue. In reality this is causing problems within a Python script that can be illustrated as follows: Both ping, nslookup and my Python code work fine in alpine:3.12 but not alpine:3.13 I'm using Docker Desktop 3.2.1 on Windows 10 with WSL2. |
|
I'm having the same issue. DNS is broken on Alpine 3.13 |
ghost
mentioned this issue
|
May this be a problem only for Docker rootless setups? As I can confirm the problem on a Docker rootless setup but it works on a default Docker setup. Docker rootless (20.10.5)user@host:~$ docker run --rm alpine:3.13 wget https://api.nuget.org/v3/index.json
wget: bad address 'api.nuget.org'Docker default (19.03.11)user@host:~$ docker run --rm alpine:3.13 wget https://api.nuget.org/v3/index.json
Connecting to api.nuget.org (152.199.23.209:443)
saving to 'index.json'
index.json 100% |********************************| 8543 0:00:00 ETA
'index.json' saved |
|
Hello, is any fix in plan for this ? Best regards |
|
I'd be happy to fix it if someone could tell what the problem is. To my understanding the problem is busybox Now for the original problem, with busybox nslookup actually returns Would be good to know:
Also, busybox |
|
does it work with |
|
I'm was having similar problems running Alpine 3.13 containers inside Kubernetes. While using an external DNS (e.g. After double checking my CoreDNS settings and versions, and making sure it would respond correctly for both A and AAAA records, the problem was no longer present. It seems to me the issue happens when DNS servers responds with |
and now it cannot |
can you please elaborate? what changes did you make to CoreDNS settings? |
I am not quite sure that this is really related to CoreDNS (or I did not understand it correctly), as the busybox nslookup will give both IPv6 and IPv4 adresses on some other hosts: alpine:3.13/ # nslookup github.com
Server: 192.168.65.1
Address: 192.168.65.1:53
** server can't find github.com: NXDOMAIN
Non-authoritative answer:
Name: github.com
Address: 140.82.121.4
Name: github.com
Address: 140.82.121.4
/ # nslookup google.com
Server: 192.168.65.1
Address: 192.168.65.1:53
Non-authoritative answer:
Name: google.com
Address: 2a00:1450:4005:802::200e
Name: google.com
Address: 2a00:1450:4005:802::200e
Non-authoritative answer:
Name: google.com
Address: 142.250.181.206
Name: google.com
Address: 142.250.181.206
/ # wget https://github.com
wget: bad address 'github.com'
/ # wget https://google.com
Connecting to google.com (216.58.213.206:443)
Connecting to www.google.com (172.217.19.68:443)
saving to 'index.html'
index.html 100% |***********************************************************************************************************************************************| 14400 0:00:00 ETA
'index.html' savedalpine:3.12/ # nslookup github.com
Server: 192.168.65.1
Address: 192.168.65.1:53
** server can't find github.com: NXDOMAIN
Non-authoritative answer:
Name: github.com
Address: 140.82.121.4
Name: github.com
Address: 140.82.121.4
/ # nslookup google.com
Server: 192.168.65.1
Address: 192.168.65.1:53
Non-authoritative answer:
Name: google.com
Address: 2a00:1450:4005:803::200e
Name: google.com
Address: 2a00:1450:4005:803::200e
Non-authoritative answer:
Name: google.com
Address: 142.250.181.206
Name: google.com
Address: 142.250.181.206
/ # wget https://github.com
Connecting to github.com (140.82.121.4:443)
saving to 'index.html'
index.html 100% |***********************************************************************************************************************************************| 208k 0:00:00 ETA
'index.html' saved
/ # rm index.html
/ # wget https://google.com
Connecting to google.com (216.58.213.206:443)
Connecting to www.google.com (216.58.213.228:443)
saving to 'index.html'
index.html 100% |***********************************************************************************************************************************************| 13609 0:00:00 ETA
'index.html' saved
@ncopa, to answer your question, The use case is any download while building container images based on alpine might fails when downloading additional parts. The list of domains I came across contains for example github.com and dl-3.alpinelinux.org. In our case we are using the $ uname -a
Linux 096ee58fe6f6 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 Linux
$ docker --version
Docker version 19.03.12, build 48a66213fe
$ dpkg -l libseccomp2
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=================-======================-============-============================================
ii libseccomp2:amd64 2.5.1-1ubuntu1~20.04.1 amd64 high level interface to Linux seccomp filter
|
|
I'm pretty sure it is not related to CoreDNS. I'm facing the issue in Docker-for-Mac, Barebone rootless docker on ubuntu as well when running in a k8s cluster. These do not all use a CoreDNS setup. Any other image is not facing this issue. e.g. Ubuntu, debian, busybox. |
|
It is not related to CoreDNS specifically, but to the DNS server returning the incorrect error code for a non existing record, e.g. NXDOMAIN errors instead of NODATA. In my case it was related to CoreDNS since this is the DNS server I'm using, I believe Docker-for-Mac has its own DNS server too, although I'm not sure and don't have a mac to test this. Could you post the output for the following commands? These should test DNS resolutions against both DNS providers: default (nameserver entry in /etc/resolv.conf) and Google's public one. I believe at least one of the records on the default DNS server will return NXDOMAIN. @lrascao I missed the notification for your message, sorry. CoreDNS versions prior to 1.7 had some issues with autopath + cache plugins, a workaround was to "disable" IPv6 resolution by returning a fixed response for every AAAA request, I still had this setting on my CoreDNS Corefile since I do not have IPv6 enabled on my environment. Removing the workaround and making sure CoreDNS is up-to-date was the change I made that solved the resolution issues with Alpine 3.13 for me. |
|
Google works fine, Github is having issues on AAAA (ipv6). Or when the $ docker run --rm -it alpine sh
/ # nslookup -type=A google.com
Server: 192.168.65.5
Address: 192.168.65.5:53
Non-authoritative answer:
Name: google.com
Address: 216.58.214.14
/ # nslookup -type=AAAA google.com
Server: 192.168.65.5
Address: 192.168.65.5:53
Non-authoritative answer:
Name: google.com
Address: 2a00:1450:400e:802::200e
/ # nslookup -type=AAAA google.com 8.8.8.8
Server: 8.8.8.8
Address: 8.8.8.8:53
Non-authoritative answer:
Name: google.com
Address: 2a00:1450:400e:802::200e
/ # nslookup -type=A google.com 8.8.8.8
Server: 8.8.8.8
Address: 8.8.8.8:53
Non-authoritative answer:
Name: google.com
Address: 216.58.214.14
/ # nslookup -type=A github.com
Server: 192.168.65.5
Address: 192.168.65.5:53
Non-authoritative answer:
Name: github.com
Address: 140.82.121.4
/ # nslookup -type=AAAA github.com
Server: 192.168.65.5
Address: 192.168.65.5:53
Non-authoritative answer:
*** Can't find github.com: No answer
/ # nslookup -type=AAAA github.com 8.8.8.8
Server: 8.8.8.8
Address: 8.8.8.8:53
Non-authoritative answer:
*** Can't find github.com: No answer
/ # nslookup github.com
Server: 192.168.65.5
Address: 192.168.65.5:53
Non-authoritative answer:
Name: github.com
Address: 140.82.121.4
Non-authoritative answer:
*** Can't find github.com: No answer |
|
Sorry @marcofranssen I forgot to add the exit code to the commands, I believe all those returned 0 (success), right? Adding to the history here, I believe this is related to https://gitlab.alpinelinux.org/alpine/aports/-/issues/11879 which should be fixed on https://git.musl-libc.org/cgit/musl/commit/?id=5cf1ac2443ad0dba263559a3fe043d929e0e5c4c (released on musl-1.2.1). Could it be possible that the affected images are still using an older version of musl? |
|
Indeed responds with a $ docker run --rm -it alpine sh
/ # nslookup github.com ; echo $?
Server: 192.168.65.5
Address: 192.168.65.5:53
Non-authoritative answer:
Name: github.com
Address: 140.82.121.4
Non-authoritative answer:
*** Can't find github.com: No answer
0
/ # apk list | grep musl-
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.13/main: No such file or directory
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.13/community: No such file or directory
musl-1.2.2-r0 x86_64 {musl} (MIT) [installed]
musl-utils-1.2.2-r0 x86_64 {musl} (MIT BSD GPL2+) [installed] |
|
Got the same problem on 3.13. It looks like all network tools prefer using the "-6" option in this version. |
…1034) Apparently in older linux kernels or docker installs (the alpine bug report isn't sure which), DNS sometimes doesn't work, and we can't build this image since github.com fails to resolve in the second line of this RUN. In Alpine 3.12, unzip doesn't create the directory automatically, so we need to make it ahead of time. Alpine 3.12 EOLs May 1, 2022, we should update by then, decide that we aren't concerned with older kernels (or docker installs), or pick a different image for this proxy. alpinelinux/docker-alpine#155 See #1119 for later followup.
|
A quick update which might add a little further insight. The issue that I've encountered personally appears to be when accessing specific domains via IPv6 with Alpine 3.13 upwards; I've tested Alpine 3.13, 3.14 and 3.15. Alpine 3.12 and earlier releases do not have this specific issue. The problem only affects some of my environments (e.g. Windows 10 running Docker Desktop with an Alpine 3.13/3.14/3.15 base image for my project). Some of my other environments work absolutely fine (e.g. Amazon Linux using Alpine 3.13/3.14/3.15 base images or my Windows machine using alternative base images such as Ubuntu / Debian). Alpine 3.12 and earlier still work fine on my Windows 10 machine. It's all the same problem but for me it is when accessing www.worldcubeassociation.org I have a simple Python script that downloads a zip file from that website but on my Windows laptop it fails due to what appears to be name resolution issue; original stack trace is included in my earlier comment but occurs in Alpine 3.13, 3.14 and 3.15. To allow myself to upgrade to Python 3.10 + Alpine 3.15, I've implemented a dirty hack in my Python project to force IPv4 when running on Windows. I test the project code on my Windows machine (i.e. IPv4) then deploy to AWS which bypasses the hack and thus uses IPv6 (if available). For anyone interested you can see the Python code that downloads the zip file from www.worldcubeassociation.org and the hack to force IPv4 on Windows: https://github.com/Logiqx/wca-db/tree/master/python The fix suggested by dogi (installing the ip6tables package) does not work for my project. |
|
I have run into this issue a few times on Alpine 3.12+ images using Docker for Windows, building multi-arch images with buildx, or even just running the base image. I would get My workaround is adding I found that this helps, not sure why. Maybe it added another way to resolve the address. This then would allow curl and wget to resolve the host the first time, but only if there was no redirect (301, 302) from the original domain, but can be resolved again using I don't think this is an ipv4 / ipv6 issue, to me it seems to be an issue with the resolver and repeated requests will resolve the address. Now whether this lies in the Alpine image or in Docker I am not sure. |
|
mark |
|
I've been running into roughly the same problem. In my I'm trying to install a dotnet tool (dotnet-monitor) and I couldn't resolve api.nuget.org I tried:
I ended up manually adding my local dns to 172.24.112.1 <- this is there by default. So, whatever that points to seems to cause problems? |
|
I'm experiencing the same issue and I am running busybox in a Docker rootless, when running towards the default Docker rootless DNS resolving part. When pointing out an external DNS in the nslookup command, it actually works in Docker rootless. When running in "normal" Docker, both commands below work. # using 1.1.1.1
docker run --rm golang:alpine /bin/busybox nslookup -debug github.com 1.1.1.1
# using default
docker run --rm golang:alpine /bin/busybox nslookup -debug github.comOutput is like this: |
|
apk add bind-tools |
|
@mj520 this does not help us, because our application still fails in DNS resolution. In our case we are also receiving NXDOMAIN / SERVFAIL for AAAA type records from our Consul cluster. |
@j4ym0 on 3.16, retrying wget 10 times doesn't fix the DNS issue after upgrading from BusyBox wget using I'm on Debian 11 on WSL2. |
|
same issue. After the docker daemon runs for a period of time, alpine's network will be broken. / # ping -4 10.0.0.1
PING 10.0.0.1 (10.0.0.1): 56 data bytes
^C
--- 10.0.0.1 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss
/ # traceroute github.com
traceroute to github.com (20.205.243.166), 30 hops max, 46 byte packets
1 172.17.0.1 (172.17.0.1) 0.076 ms 0.007 ms 0.003 ms
2 * * *
3 * * *
4 * * *
5 * * */ # apk add iputils
fetch https://dl-cdn.alpinelinux.org/alpine/v3.17/main/x86_64/APKINDEX.tar.gz
48BB57250C7F0000:error:0A000126:SSL routines:ssl3_read_n:unexpected eof while reading:ssl/record/rec_layer_s3.c:320:
ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.17/main: Permission denied
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.17/main: No such file or directory
fetch https://dl-cdn.alpinelinux.org/alpine/v3.17/community/x86_64/APKINDEX.tar.gz
48BB57250C7F0000:error:0A000126:SSL routines:ssl3_read_n:unexpected eof while reading:ssl/record/rec_layer_s3.c:320:
ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.17/community: Permission denied
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.17/community: No such file or directory
ERROR: unable to select packages:
iputils (no such package):
required by: world[iputils] |
Using the latest alpine
3.13I'm unable to reach github.Have tried this on Docker for Mac and a ubuntu 20.04 with rootless docker.
See here some things I tried to reach that succeeded.
See here
github.comwhich fails.For completeness I also checked with
debian:buster-slimwhich succeeds.The text was updated successfully, but these errors were encountered: