Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DNSError when providing nameservers to DNSResolver #15

Closed
maartendraijer opened this issue Jun 30, 2016 · 8 comments
Closed

DNSError when providing nameservers to DNSResolver #15

maartendraijer opened this issue Jun 30, 2016 · 8 comments

Comments

@maartendraijer
Copy link

Hi!

Question. I'm trying to fetch the name servers of a specific domain name. This works when I don't provide nameservers to the DNSResolver:

import asyncio
import aiodns

loop = asyncio.get_event_loop()
resolver = aiodns.DNSResolver(loop=loop)
f = resolver.query('test.de', 'NS')
result = loop.run_until_complete(f)
print(result)

This results in:

[ares_query_ns_result(host='dns3.iwelt-ag.de', ttl=None), ares_query_ns_result(host='dns2.iwelt-ag.net', ttl=None), ares_query_ns_result(host='dns.iwelt-ag.net', ttl=None)]

But, when I do this:

import asyncio
import aiodns

loop = asyncio.get_event_loop()
resolver = aiodns.DNSResolver(loop=loop, nameservers=['81.91.164.5'])
f = resolver.query('test.de', 'NS')
result = loop.run_until_complete(f)
print(result)

This is the result:

aiodns.error.DNSError: (1, 'DNS server returned answer with no data')

Though when I capture the packets I see that I'm getting the results back from the name server:

 sudo tcpdump -i en0 -AAA -vvv host 81.91.164.5

tcpdump: listening on en0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:40:30.518946 IP (tos 0x0, ttl 64, id 23865, offset 0, flags [none], proto UDP (17), length 53)
    192.168.178.10.55500 > f.nic.de.domain: [udp sum ok] 58234+ NS? test.de. (25)
.Y3.;/..V?. ..E..5]9..@..k...
Q[.....5.!...z...........test.de.....
21:40:30.532423 IP (tos 0x0, ttl 57, id 27877, offset 0, flags [none], proto UDP (17), length 146)
    f.nic.de.domain > 192.168.178.10.55500: [udp sum ok] 58234- q: NS? test.de. 0/3/1 ns: test.de. [1d] NS dns.iwelt-ag.net., test.de. [1d] NS dns2.iwelt-ag.net., test.de. [1d] NS dns3.iwelt-ag.de. ar: dns3.iwelt-ag.de. [1d] A 85.236.41.196 (118)
..V?. .Y3.;/..E...l...9..bQ[.....
.5...~...z...........test.de.............Q....dns.iwelt-ag.net.........Q....dns2.)........Q....dns3.iwelt-ag...V......Q...U.).

Am I missing something, or is this a bug somewhere in aiodns or pycares?

@saghul
Copy link
Owner

saghul commented Jun 30, 2016

I added the following test in pycares: saghul/pycares@9689d86

It passes everywhere except on PyPy (for other reasons). So I guess it's the underlying c-ares library. Can you try to install or compile adig (included with c-ares, it's like dig but based on c-ares) and give it a try? Or try another nameserver just in case too?

@maartendraijer
Copy link
Author

maartendraijer commented Jul 1, 2016

Thanks for your reply @saghul.

Hmm, when I run this:

import asyncio
import aiodns

loop = asyncio.get_event_loop()
resolver = aiodns.DNSResolver(loop=loop, nameservers=['8.8.8.8'])
f = resolver.query('test.de', 'NS')
result = loop.run_until_complete(f)
print(result)

I get a result:

[ares_query_ns_result(host='dns3.iwelt-ag.de', ttl=None), ares_query_ns_result(host='dns2.iwelt-ag.net', ttl=None), ares_query_ns_result(host='dns.iwelt-ag.net', ttl=None)

These are the packets:

tcpdump: listening on en0, link-type EN10MB (Ethernet), capture size 262144 bytes
08:19:45.295169 IP (tos 0x0, ttl 64, id 43082, offset 0, flags [none], proto UDP (17), length 53)
    192.168.50.68.60405 > google-public-dns-a.google.com.domain: [udp sum ok] 64917+ NS? test.de. (25)
...6....V?. ..E..5.J..@..q..2D.......5.!...............test.de.....
08:19:45.323672 IP (tos 0x0, ttl 48, id 41764, offset 0, flags [none], proto UDP (17), length 130)
    google-public-dns-a.google.com.domain > 192.168.50.68.60405: [udp sum ok] 64917 q: NS? test.de. 3/0/0 test.de. [9m59s] NS dns3.iwelt-ag.de., test.de. [9m59s] NS dns2.iwelt-ag.net., test.de. [9m59s] NS dns.iwelt-ag.net. (102)
..V?. ...6....E....$..0..J......2D.5...n...............test.de..............W...dns3.iwelt-ag...........W...dns2.iwelt-ag.net..........W...dns.F

Though when I try any of these servers (from the ADDITIONAL SECTION), it fails:

; <<>> DiG 9.8.3-P1 <<>> de @a.root-servers.net NS
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62080
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 6, ADDITIONAL: 10
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;de.                            IN      NS

;; AUTHORITY SECTION:
de.                     172800  IN      NS      a.nic.de.
de.                     172800  IN      NS      f.nic.de.
de.                     172800  IN      NS      l.de.net.
de.                     172800  IN      NS      n.de.net.
de.                     172800  IN      NS      s.de.net.
de.                     172800  IN      NS      z.nic.de.

;; ADDITIONAL SECTION:
a.nic.de.               172800  IN      A       194.0.0.53
f.nic.de.               172800  IN      A       81.91.164.5
l.de.net.               172800  IN      A       77.67.63.105
n.de.net.               172800  IN      A       194.146.107.6
s.de.net.               172800  IN      A       195.243.137.26
z.nic.de.               172800  IN      A       194.246.96.1
a.nic.de.               172800  IN      AAAA    2001:678:2::53
f.nic.de.               172800  IN      AAAA    2a02:568:0:2::53
l.de.net.               172800  IN      AAAA    2001:668:1f:11::105
n.de.net.               172800  IN      AAAA    2001:67c:1011:1::53

;; Query time: 30 msec
;; SERVER: 198.41.0.4#53(198.41.0.4)
;; WHEN: Fri Jul  1 08:21:32 2016
;; MSG SIZE  rcvd: 334

But what tcpdump shows me is that results are coming back from any of them:

 sudo tcpdump -i en0 -AAA -vvv host 194.246.96.1

tcpdump: listening on en0, link-type EN10MB (Ethernet), capture size 262144 bytes
08:27:39.786882 IP (tos 0x0, ttl 64, id 10852, offset 0, flags [none], proto UDP (17), length 53)
    192.168.50.68.63445 > z.nic.de.domain: [udp sum ok] 55343+ NS? test.de. (25)
...6....V?. ..E..5*d..@.:p..2D..`....5.!.<./...........test.de.....
08:27:39.804282 IP (tos 0x0, ttl 55, id 32999, offset 0, flags [none], proto UDP (17), length 146)
    z.nic.de.domain > 192.168.50.68.63445: [udp sum ok] 55343- q: NS? test.de. 0/3/1 ns: test.de. [1d] NS dns.iwelt-ag.net., test.de. [1d] NS dns2.iwelt-ag.net., test.de. [1d] NS dns3.iwelt-ag.de. ar: dns3.iwelt-ag.de. [1d] A 85.236.41.196 (118)
..V?. ...6....E.......7.....`...2D.5...~.../...........test.de.............Q....dns.iwelt-ag.net.........Q....dns2.)........Q....dns3.iwelt-ag...V......Q...U.).

And using dig things are working too:

 dig test.de NS @81.91.164.5   

; <<>> DiG 9.8.3-P1 <<>> test.de NS @81.91.164.5
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64228
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 3, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;test.de.                       IN      NS

;; AUTHORITY SECTION:
test.de.                86400   IN      NS      dns.iwelt-ag.net.
test.de.                86400   IN      NS      dns2.iwelt-ag.net.
test.de.                86400   IN      NS      dns3.iwelt-ag.de.

;; ADDITIONAL SECTION:
dns3.iwelt-ag.de.       86400   IN      A       85.236.41.196

;; Query time: 38 msec
;; SERVER: 81.91.164.5#53(81.91.164.5)
;; WHEN: Fri Jul  1 08:31:14 2016
;; MSG SIZE  rcvd: 118

Also I tried some other tlds (com & nl), and those are not working either. Nasty.

I can't seem to find anything about adig. I'm on OS X by the way.

@saghul
Copy link
Owner

saghul commented Jul 1, 2016

Hum, I really don't know what's going on :-( I can only speculate it's somewhere deep down c-ares, since using a different nameserver works...

@saghul
Copy link
Owner

saghul commented Jul 1, 2016

@ioc32 hey mate, does any of this (the DNS traces) make any sense to you?

@ioc32
Copy link

ioc32 commented Jul 1, 2016

@saghul @maartendraijer I do not currently have a virtualenv where I can test this, but what you're experiencing may be related to the underlying subtleties in the DNS queries you are firing and how strict the library backend is when parsing them.

Let me elaborate:

DNS is an extremely lenient protocol and will allow for "some" answers being sent back as responses to queries that are not strictly correct.

In this case, you get the expected answers from aiodns/pycares when you pass a caching resolver to the DNSResolver class, but you do not when you pass an authoritative name-server (all name-servers in the additional section are of course authoritative).

I don't know how DNSResolver treats DNS responses with the RA (Recursion Available) bit set to zero.

Please have a look at the differences in the DNS flags and responses here [1], hopefully the examples there are more helpful than my explanation :)

[1]
iortiz@rigel:~
$ dig @81.91.164.5 +norec test.de ns

; <<>> DiG 9.8.3-P1 <<>> @81.91.164.5 +norec test.de ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19290
;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 3, ADDITIONAL: 1

;; QUESTION SECTION:
;test.de. IN NS

;; AUTHORITY SECTION:
test.de. 86400 IN NS dns.iwelt-ag.net.
test.de. 86400 IN NS dns2.iwelt-ag.net.
test.de. 86400 IN NS dns3.iwelt-ag.de.

;; ADDITIONAL SECTION:
dns3.iwelt-ag.de. 86400 IN A 85.236.41.196

;; Query time: 29 msec
;; SERVER: 81.91.164.5#53(81.91.164.5)
;; WHEN: Fri Jul 1 12:10:32 2016
;; MSG SIZE rcvd: 118

iortiz@rigel:~
$ dig @81.91.164.5 test.de ns

; <<>> DiG 9.8.3-P1 <<>> @81.91.164.5 test.de ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 32736
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 3, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;test.de. IN NS

;; AUTHORITY SECTION:
test.de. 86400 IN NS dns.iwelt-ag.net.
test.de. 86400 IN NS dns2.iwelt-ag.net.
test.de. 86400 IN NS dns3.iwelt-ag.de.

;; ADDITIONAL SECTION:
dns3.iwelt-ag.de. 86400 IN A 85.236.41.196

;; Query time: 30 msec
;; SERVER: 81.91.164.5#53(81.91.164.5)
;; WHEN: Fri Jul 1 12:10:36 2016
;; MSG SIZE rcvd: 118

iortiz@rigel:~
$ dig @8.8.8.8 test.de ns

; <<>> DiG 9.8.3-P1 <<>> @8.8.8.8 test.de ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14453
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;test.de. IN NS

;; ANSWER SECTION:
test.de. 599 IN NS dns2.iwelt-ag.net.
test.de. 599 IN NS dns.iwelt-ag.net.
test.de. 599 IN NS dns3.iwelt-ag.de.

;; Query time: 29 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Jul 1 12:10:49 2016
;; MSG SIZE rcvd: 102

@thehesiod
Copy link

thehesiod commented Oct 2, 2016

btw we're hitting this too when used w/ aiohttp with requests to water.weather.gov:80

update: seems to be resolved

@thehesiod
Copy link

I added the following test in pycares: saghul/pycares@9689d86

It passes everywhere except on PyPy (for other reasons). So I guess it's the underlying c-ares library. Can you try to install or compile adig (included with c-ares, it's like dig but based on c-ares) and give it a try? Or try another nameserver just in case too?

could you try this on debian stretch with python3.6? I tried both the pycares wheel and installing from source.

After building c-ares from source@master, here are the results comparing dig with adig

bash-4.4# dig @81.91.164.5 test.de

; <<>> DiG 9.10.3-P4-Debian <<>> @81.91.164.5 test.de
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49319
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 3, ADDITIONAL: 2
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;test.de.			IN	A

;; AUTHORITY SECTION:
test.de.		86400	IN	NS	dns.iwelt-ag.net.
test.de.		86400	IN	NS	dns2.iwelt-ag.net.
test.de.		86400	IN	NS	dns3.iwelt-ag.de.

;; ADDITIONAL SECTION:
dns3.iwelt-ag.de.	86400	IN	A	85.236.41.196

;; Query time: 192 msec
;; SERVER: 81.91.164.5#53(81.91.164.5)
;; WHEN: Wed Nov 14 23:13:23 UTC 2018
;; MSG SIZE  rcvd: 129
bash-4.4# bin/adig -s 81.91.164.5 -t NS test.de
DNS server returned answer with no data
id: 10118
flags: qr rd 
opcode: QUERY
rcode: NOERROR
Questions:
	test.de        .		NS
Answers:
NS records:
	test.de        .	86400	NS	dns.iwelt-ag.net.
	test.de        .	86400	NS	dns2.iwelt-ag.net.
	test.de        .	86400	NS	dns3.iwelt-ag.de.
Additional records:
	dns3.iwelt-ag.de.	86400	A	85.236.41.196

@saghul
Copy link
Owner

saghul commented Jan 17, 2019

Closing since there is not much we can do on our side. c-ares is doing the heavy lifting. That said, pycates 3.0.0 (currently in beta) bumped c-ares to version 1.15.0, maybe some bug related to this got fixed.

@saghul saghul closed this as completed Jan 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants