Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DNS Resolution fails after updating libc6 and/or libc-bin #2873

Open
ralphyz opened this issue Jan 23, 2018 · 14 comments
Open

DNS Resolution fails after updating libc6 and/or libc-bin #2873

ralphyz opened this issue Jan 23, 2018 · 14 comments
Labels

Comments

@ralphyz
Copy link

@ralphyz ralphyz commented Jan 23, 2018

Your Windows build number: (Type ver at a Windows Command Prompt)
Microsoft Windows [Version 10.0.16299.125]

What you're doing and what's happening: (Copy&paste specific commands and their output, or include screen shots)
After updating libc6 and/or libc-bin apt, wget, curl, etc. quit working because they can no longer resolve hostnames. I followed the guide here:
https://www.kali.org/tutorials/kali-on-the-windows-subsystem-for-linux/

I have tried both of these dockers with the same effects:
kalilinux/kali-linux-docker
jasonchaffee/kali-linux

Steps to cause the problem:
root@W10:/tmp# apt install libc-bin
or
root@W10:/tmp# apt install libc6

After either of those packages are updated, then anything needing a DNS resolution fails:
Showing that this works:

root@W10:/tmp# apt search libc-bin
Sorting... Done
Full Text Search... Done
libc-bin/kali-rolling 2.26-4 amd64 [upgradable from: 2.25-5]
  GNU C Library: Binaries

root@W10:/tmp# wget www.google.com
--2018-01-23 16:10:33--  http://www.google.com/
Resolving www.google.com (www.google.com)... 172.217.2.36, 2607:f8b0:4002:808::2004
Connecting to www.google.com (www.google.com)|172.217.2.36|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4749 (4.6K) [text/html]
Saving to: 'index.html'

index.html                                                  100%[========================================================================================================================================>]   4.64K  --.-KB/s    in 0s

2018-01-23 16:10:33 (113 MB/s) - 'index.html' saved [11033]

Now, updating libc-bin (which includes libc6)

root@W10:/tmp# apt install libc-bin
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libc6
Suggested packages:
  glibc-doc libc-l10n locales
Recommended packages:
  manpages
The following packages will be upgraded:
  libc-bin libc6
2 upgraded, 0 newly installed, 0 to remove and 35 not upgraded.
Need to get 3632 kB of archives.
After this operation, 285 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://archive-7.kali.org/kali kali-rolling/main amd64 libc6 amd64 2.26-4 [2868 kB]
Get:2 http://archive-7.kali.org/kali kali-rolling/main amd64 libc-bin amd64 2.26-4 [764 kB]
Fetched 3632 kB in 3s (1137 kB/s)
Preconfiguring packages ...
(Reading database ... 9041 files and directories currently installed.)
Preparing to unpack .../libc6_2.26-4_amd64.deb ...
/var/lib/dpkg/tmp.ci/preinst: 1: /var/lib/dpkg/tmp.ci/preinst: head: not found
/var/lib/dpkg/tmp.ci/preinst: 1: /var/lib/dpkg/tmp.ci/preinst: head: not found
/var/lib/dpkg/tmp.ci/preinst: 1: /var/lib/dpkg/tmp.ci/preinst: head: not found
/var/lib/dpkg/tmp.ci/preinst: 1: /var/lib/dpkg/tmp.ci/preinst: head: not found
/var/lib/dpkg/tmp.ci/preinst: 1: /var/lib/dpkg/tmp.ci/preinst: head: not found
Checking for services that may need to be restarted...
Checking init scripts...
Unpacking libc6:amd64 (2.26-4) over (2.25-5) ...
Setting up libc6:amd64 (2.26-4) ...
Checking for services that may need to be restarted...
Checking init scripts...
Nothing to restart.
(Reading database ... 9041 files and directories currently installed.)
Preparing to unpack .../libc-bin_2.26-4_amd64.deb ...
Unpacking libc-bin (2.26-4) over (2.25-5) ...
Setting up libc-bin (2.26-4) ...
W: http: aptMethod::Configuration: could not load seccomp policy: Invalid argument
W: http: aptMethod::Configuration: could not load seccomp policy: Invalid argument

And finally, showing wget doesn't work anymore, nor does anything requiring name resolution (apt, traceroute, ping):

root@W10:/tmp# wget www.google.com
--2018-01-23 16:11:15--  http://www.google.com/
Resolving www.google.com (www.google.com)... failed: Temporary failure in name resolution.
wget: unable to resolve host address 'www.google.com'
root@W10:/tmp#

What's wrong / what should be happening instead:
All name resolution should be working. I can still ping internal IP Addresses, but I cannot reach anything external after updating libc6 or libc-bin.

Strace of the failing command, if applicable: (If some_command is failing, then run strace -o some_command.strace -f some_command some_args, and link the contents of some_command.strace in a gist here)
strace -o wget.strace -f wget google.com

https://gist.github.com/ralphyz/c5bbfc575738d67eba84a0a9f1ce3d22

@Brian-Perkins

This comment has been minimized.

Copy link
Collaborator

@Brian-Perkins Brian-Perkins commented Jan 23, 2018

The strace shows the nameservers 192.168.1.1 and 8.8.8.8 both being tried and not responding. Do those addresses seem reasonable in your setup? It would be interesting to compare this against the working scenario.

@ralphyz

This comment has been minimized.

Copy link
Author

@ralphyz ralphyz commented Jan 24, 2018

Yes - that is the contents of my /etc/resolv.conf. It is the information that existed before libc6 was upgraded - demonstrated working prior to the update.

@ralphyz

This comment has been minimized.

Copy link
Author

@ralphyz ralphyz commented Jan 24, 2018

apt update no longer works, traceroute <url> no longer works. wget <url> no longer works. Basically, anything that relies on DNS fails. I can wget the IP Address of a URL, but not using the DNS name.

@sunilmut

This comment has been minimized.

Copy link
Member

@sunilmut sunilmut commented Jan 24, 2018

@ralphyz - I think what Brian is suggesting by It would be interesting to compare this against the working scenario. is to gather a strace from the working scenario and then compare it against the non-working setup to note interesting differences. If you can upload the strace from the working setup, we can take a look.

@sunilmut sunilmut added the network label Jan 24, 2018
@therealkenc

This comment has been minimized.

Copy link
Collaborator

@therealkenc therealkenc commented Jan 24, 2018

It would be interesting to compare this against the working scenario.

Only difference I can see is SOCK_CLOEXEC. Not much else to work with if the OP can get to the Interwebs via IP address. The sequence is that short.

2.26 (from the linked strace)

397   socket(AF_INET, SOCK_DGRAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 4
397   connect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("8.8.8.8")}, 16) = 0
397   gettimeofday({tv_sec=1516743090, tv_usec=148731}, NULL) = 0
397   poll([{fd=4, events=POLLOUT}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
397   sendmmsg(4, [{msg_hdr={msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\306#\1\0\0\1\0\0\0\0\0\0\6google\3com\0\0\1\0\1", iov_len=28}], msg_iovlen=1, msg_controllen=0, msg_flags=MSG_TRUNC|MSG_DONTWAIT|MSG_WAITALL|MSG_FIN|MSG_CONFIRM|MSG_NOSIGNAL|MSG_BATCH|MSG_CMSG_CLOEXEC|0x80e00000}, msg_len=28}, {msg_hdr={msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\236_\1\0\0\1\0\0\0\0\0\0\6google\3com\0\0\34\0\1", iov_len=28}], msg_iovlen=1, msg_controllen=0, msg_flags=MSG_DONTWAIT|MSG_EOR|MSG_WAITALL|MSG_FIN|MSG_CONFIRM|MSG_NOSIGNAL|MSG_BATCH|0x92f20000}, msg_len=28}], 2, MSG_NOSIGNAL) = 2
397   poll([{fd=4, events=POLLIN}], 1, 5000) = 0 (Timeout)

2.23 (random nslookup clip from my Ubuntu 16.04.03 WSL)

socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("8.8.8.8")}, 16) = 0
gettimeofday({1516749817, 22967}, NULL) = 0
poll([{fd=3, events=POLLOUT}], 1, 0)    = 1 ([{fd=3, revents=POLLOUT}])
sendmmsg(3, {{{msg_name(0)=NULL, msg_iov(1)=[{"\1\353\1\0\0\1\0\0\0\0\0\0\6github\3com\0\0\1\0\1", 28}], msg_controllen=0, msg_flags=MSG_DONTROUTE|MSG_EOR|MSG_FIN|MSG_CONFIRM|MSG_RST|MSG_NOSIGNAL|MSG_WAITFORONE|MSG_FASTOPEN|MSG_CMSG_CLOEXEC|0x80120010}, 28}, {{msg_name(0)=NULL, msg_iov(1)=[{"\3245\1\0\0\1\0\0\0\0\0\0\6github\3com\0\0\34\0\1", 28}], msg_controllen=0, msg_flags=MSG_OOB|MSG_DONTROUTE|MSG_CTRUNC|0x10}, 28}}, 2, MSG_NOSIGNAL) = 2
poll([{fd=3, events=POLLIN}], 1, 5000)  = 1 ([{fd=3, revents=POLLIN}])
@therealkenc

This comment has been minimized.

Copy link
Collaborator

@therealkenc therealkenc commented Jan 24, 2018

Nah wait... The message being sent is structurally different. Google might be just refusing to reply. Unless SOCK_CLOEXEC rings a really loud bell, better to back up and look at the install fails (which will almost certainly surface eventually in a future Ubuntu release).

@ralphyz

This comment has been minimized.

Copy link
Author

@ralphyz ralphyz commented Jan 24, 2018

@ralphyz

This comment has been minimized.

Copy link
Author

@ralphyz ralphyz commented Jan 24, 2018

I did an apt update with an strace:
strace -o apt_update(working).strace -f apt update
Gist of strace here: https://gist.github.com/ralphyz/ae3f50817ba4c0b6d9f13d428f73d298

Then I updated libc6:

root@W10:/tmp# apt install libc6
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libc-bin
Suggested packages:
  glibc-doc libc-l10n locales
Recommended packages:
  manpages
The following packages will be upgraded:
  libc-bin libc6
2 upgraded, 0 newly installed, 0 to remove and 35 not upgraded.
Need to get 3632 kB of archives.
After this operation, 285 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://archive-7.kali.org/kali kali-rolling/main amd64 libc6 amd64 2.26-4 [2868 kB]
Get:2 http://archive-7.kali.org/kali kali-rolling/main amd64 libc-bin amd64 2.26-4 [764 kB]
Fetched 3632 kB in 3s (1445 kB/s)
Preconfiguring packages ...
(Reading database ... 9050 files and directories currently installed.)
Preparing to unpack .../libc6_2.26-4_amd64.deb ...
/var/lib/dpkg/tmp.ci/preinst: 1: /var/lib/dpkg/tmp.ci/preinst: head: not found
/var/lib/dpkg/tmp.ci/preinst: 1: /var/lib/dpkg/tmp.ci/preinst: head: not found
/var/lib/dpkg/tmp.ci/preinst: 1: /var/lib/dpkg/tmp.ci/preinst: head: not found
/var/lib/dpkg/tmp.ci/preinst: 1: /var/lib/dpkg/tmp.ci/preinst: head: not found
/var/lib/dpkg/tmp.ci/preinst: 1: /var/lib/dpkg/tmp.ci/preinst: head: not found
Checking for services that may need to be restarted...
Checking init scripts...
Unpacking libc6:amd64 (2.26-4) over (2.25-5) ...
Setting up libc6:amd64 (2.26-4) ...
Checking for services that may need to be restarted...
Checking init scripts...
Nothing to restart.
(Reading database ... 9050 files and directories currently installed.)
Preparing to unpack .../libc-bin_2.26-4_amd64.deb ...
Unpacking libc-bin (2.26-4) over (2.25-5) ...
Setting up libc-bin (2.26-4) ...
W: http: aptMethod::Configuration: could not load seccomp policy: Invalid argument
W: http: aptMethod::Configuration: could not load seccomp policy: Invalid argument

Then I ran the same apt command:
strace -o "apt_update(not-working).strace" -f apt update
Gist of strace here: https://gist.github.com/ralphyz/a3959fe16805218a8474d79e388d61ce

Output from command:

root@W10:/tmp# strace -o "apt_update(not-working).strace" -f apt update
Err:1 http://http.kali.org/kali kali-rolling InRelease
  Temporary failure resolving 'http.kali.org'
Reading package lists... Done
Building dependency tree
Reading state information... Done
35 packages can be upgraded. Run 'apt list --upgradable' to see them.
W: http: aptMethod::Configuration: could not load seccomp policy: Invalid argument
W: Failed to fetch http://http.kali.org/kali/dists/kali-rolling/InRelease  Temporary failure resolving 'http.kali.org'
W: Some index files failed to download. They have been ignored, or old ones used instead.
@ralphyz

This comment has been minimized.

Copy link
Author

@ralphyz ralphyz commented Jan 24, 2018

@therealkenc the error shown in my original output was due to head not being found on that lightweight OS version I was trying on that run. I don't get that error usually.

root@W10:/tmp# apt install libc6
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libc-bin
Suggested packages:
  glibc-doc libc-l10n locales
Recommended packages:
  manpages
The following packages will be upgraded:
  libc-bin libc6
2 upgraded, 0 newly installed, 0 to remove and 38 not upgraded.
Need to get 3632 kB of archives.
After this operation, 285 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://archive-7.kali.org/kali kali-rolling/main amd64 libc6 amd64 2.26-4 [2868 kB]
Get:2 http://archive-7.kali.org/kali kali-rolling/main amd64 libc-bin amd64 2.26-4 [764 kB]
Fetched 3632 kB in 3s (1332 kB/s)
Preconfiguring packages ...
(Reading database ... 11066 files and directories currently installed.)
Preparing to unpack .../libc6_2.26-4_amd64.deb ...
Checking for services that may need to be restarted...
Checking init scripts...
Unpacking libc6:amd64 (2.26-4) over (2.25-5) ...
Setting up libc6:amd64 (2.26-4) ...
Checking for services that may need to be restarted...
Checking init scripts...
Nothing to restart.
(Reading database ... 11066 files and directories currently installed.)
Preparing to unpack .../libc-bin_2.26-4_amd64.deb ...
Unpacking libc-bin (2.26-4) over (2.25-5) ...
Setting up libc-bin (2.26-4) ...
W: http: aptMethod::Configuration: could not load seccomp policy: Invalid argument
W: http: aptMethod::Configuration: could not load seccomp policy: Invalid argument

and immediately after the install, I test to see if my DNS resolution is broken:

root@W10:/tmp# wget github.com
Will not apply HSTS. The HSTS database must be a regular and non-world-writable file.
ERROR: could not open HSTS store at '/root/.wget-hsts'. HSTS will be disabled.
--2018-01-24 10:40:29--  http://github.com/
Resolving github.com (github.com)... failed: Temporary failure in name resolution.
wget: unable to resolve host address 'github.com'

It is :(

@sunilmut

This comment has been minimized.

Copy link
Member

@sunilmut sunilmut commented Jan 25, 2018

@ralphyz - I compared the two traces and there is not much interesting from the WSL perspective. There are few additional flags that are being passed (by libc6) in sendmmsg but, those shouldn't play much of a role here. I was not able to repro the issue on my WSL setup (but, that is running Ubuntu, not Kali). In general, the interesting thing (and as also Ken points out) is that the DNS query data format seems to have changed in libc6.

As next step, I would recommend trying these steps on a native Kali setup, if you haven't already.

@therealkenc

This comment has been minimized.

Copy link
Collaborator

@therealkenc therealkenc commented Jan 26, 2018

As next step, I would recommend trying these steps on a native Kali setup, if you haven't already.

Someone very highly motivated could update /etc/apt/sources.list and try the Ubuntu 18 (Bionic) betas rather than Kali and see how it fares. Looks like it uses glibc 2.26. I mean, way more motivated than myself as of this instant anyway.

@priyadarshan

This comment has been minimized.

Copy link

@priyadarshan priyadarshan commented Dec 31, 2018

I am experiencing exact same issue with Ubuntu 18.04 and Debian 9. Using

PS C:\> [System.Environment]::OSVersion.Version
Major  Minor  Build  Revision 
10     0      17763  0   

@priyadarshan

This comment has been minimized.

Copy link

@priyadarshan priyadarshan commented Jan 4, 2019

To come back to my previous comment, I tracked the issue to having Kaspersky antivirus installed.

Even disabling it did not work. The moment it was uninstalled WSL connectivity came back as it supposed to be.

Even latest Kaspersky 11 has this issue with WSL

@transmitterdan

This comment has been minimized.

Copy link

@transmitterdan transmitterdan commented Jun 17, 2019

My system has this problem or something very similar. Would really like some help with this. The proximate cause is sendmmsg() fails and returns -1 EINVAL (Invalid argument):

Here is strace from failed DNS lookup:
pingfail.txt

And strace from working DNS lookup:
pingok.txt

The only difference between these DNS attempts is what LAN the computer is connected to. It seems that in the working case the system does not open /etc/resolv.conf. It stat's the file but doesn't open it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
You can’t perform that action at this time.