SetEDNSOption does not work as expected with dnssec enabled #11728

KlettIT · 2022-06-28T06:24:25Z

Program: dnsdist
Issue type: Bug report

Short description

When using SetEDNSOptionAction questions with dnssec (dig +dnssec) the DNS answer is empty and the status is BADVERS

Environment

Operating system: CentOS 7
Software version: 1.7.1 (Lua 5.1.4 [LuaJIT 2.0.4])
Software source: PowerDNS repository

Sample configuration

setLocal("127.0.0.20:53")
newServer({address="208.67.222.222", name="resolver1.opendns.com", order=1, pool="prodpool", reconnectOnUp=true, maxCheckFailures=3, rise=3})
newServer({address="208.67.222.222", name="resolver1.opendns.com", order=1, pool="testpool", reconnectOnUp=true, maxCheckFailures=3, rise=3})
ZonesToBeRoutedToTestPool = newSuffixMatchNode()
ZonesToBeRoutedToTestPool:add(newDNSName("test.com."))
addAction(SuffixMatchNodeRule(ZonesToBeRoutedToTestPool), SetEDNSOptionAction(20292,"ODNS\x01\x00\x00\x08\x00\x79\xbe\xef\x00\x10\xc0\xa8\x01\x37"))
addAction(SuffixMatchNodeRule(ZonesToBeRoutedToTestPool), PoolAction("testpool"))
addAction(AllRule(), PoolAction("prodpool"))

Steps to reproduce

1.) Query `test.com` without dnssec

dig @127.0.0.20 test.com A +nodnssec

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.9 <<>> @127.0.0.20 test.com A +nodnssec
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 9320
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;test.com.                      IN      A

;; ANSWER SECTION:
test.com.               3469    IN      A       67.225.146.248

;; Query time: 5 msec
;; SERVER: 127.0.0.20#53(127.0.0.20)
;; WHEN: Tue Jun 28 09:17:54 CEST 2022
;; MSG SIZE  rcvd: 53

✅ Expected A-record is returned

2.) Query `test.com` with dnssec

dig @127.0.0.20 test.com A +dnssec

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.9 <<>> @127.0.0.20 test.com A +dnssec
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: BADVERS, id: 21798
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;test.com.                      IN      A

;; Query time: 5 msec
;; SERVER: 127.0.0.20#53(127.0.0.20)
;; WHEN: Tue Jun 28 09:18:33 CEST 2022
;; MSG SIZE  rcvd: 37

❌ No data is returned and status is BADVERS

Verification

1.) Query some other domain without dnssec

dig @127.0.0.20 dnsdist.org A +nodnssec

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.9 <<>> @127.0.0.20 dnsdist.org A +nodnssec
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30651
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;dnsdist.org.                   IN      A

;; ANSWER SECTION:
dnsdist.org.            86400   IN      A       188.166.104.92

;; Query time: 73 msec
;; SERVER: 127.0.0.20#53(127.0.0.20)
;; WHEN: Tue Jun 28 09:18:53 CEST 2022
;; MSG SIZE  rcvd: 56

2.) Query the same domain with dnssec

dig @127.0.0.20 dnsdist.org A +dnssec

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.9 <<>> @127.0.0.20 dnsdist.org A +dnssec
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 54221
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;dnsdist.org.                   IN      A

;; ANSWER SECTION:
dnsdist.org.            86385   IN      A       188.166.104.92

;; Query time: 5 msec
;; SERVER: 127.0.0.20#53(127.0.0.20)
;; WHEN: Tue Jun 28 09:19:08 CEST 2022
;; MSG SIZE  rcvd: 56

✅ Expected A-record is returned with and without dnssec

Expected behaviour

Both queries regardless of the dnssec option should work when SetEDNSOption is used

Actual behaviour

As seen above the question is not answered correctly.

The text was updated successfully, but these errors were encountered:

Habbie · 2022-06-28T06:27:03Z

Can you tcpdump traffic between your dnsdist and OpenDNS?

Habbie · 2022-06-28T06:29:45Z

Also, some of your digs went to the wrong IP - 127.0.0.20. Please always provide entirely unedited information so we are certain what we are debugging.

KlettIT · 2022-06-28T06:34:40Z

Also, some of your digs went to the wrong IP - 127.0.0.20. Please always provide entirely unedited information so we are certain what we are debugging.

I have updated the issue with current outputs. I will take a tcpdump of it as well.

KlettIT · 2022-06-28T07:50:36Z

Can you tcpdump traffic between your dnsdist and OpenDNS?

Please find the tcpdump (tcpdump -i any -nn -w - -U 'host 127.0.0.20 or host 208.67.222.222' | tee dump.pcap | tcpdump -nn -r -.) attached.
dump.zip

@rgacogne: Since you have already added a defect-label: Are you aware of the issue and the reason for it? Is there a simple fix we could implement ourselves till 1.8.0 gets released?

rgacogne · 2022-06-28T07:58:39Z

Since you have already added a defect-label: Are you aware of the issue and the reason for it? Is there a simple fix we could implement ourselves till 1.8.0 gets released?

I'm not, no. I have added the label because based on your description it really looks like a bug, but I won't know for sure until I have reproduced it which will likely have to wait until tomorrow.
I did put 1.8.0 as milestone because we will fix it in the dev branch first, but a fix for such a bug will be backported to 1.7.x, to be clear.

rgacogne · 2022-06-28T08:38:36Z

It was easier than I expected so I just opened a pull request (#11729) that fixes this issue in my test. If by any chance you could test it (which involves recompiling, sorry), that would be great :)

KlettIT · 2022-06-28T09:08:25Z

Wow! Thanks for looking into it and fixing it faster than I could even look for an update of the issue 😊
We will recompile and test the fix asap.

Not directly related to this issue, but maybe you have an idea first spot so I mention it: When we were troubleshooting the issue we had pdns-rec as backend. We enabled tracing but didn't see a single log line regarding the query. Only in tcpdump we were able to spot it. So maybe there is an issue in pdns-rec as well, when such "malformed" packets arrive?

KlettIT · 2022-06-28T15:28:44Z

Sorry, we had some trouble setting up a appropriate build environment.

I can confirm your commit fixes the issue:
dig @127.0.0.20 test.com A +dnssec

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.9 <<>> @127.0.0.20 test.com A +dnssec
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20791
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;test.com.                      IN      A

;; ANSWER SECTION:
test.com.               2123    IN      A       67.225.146.248

;; Query time: 5 msec
;; SERVER: 127.0.0.20#53(127.0.0.20)
;; WHEN: Tue Jun 28 17:24:54 CEST 2022
;; MSG SIZE  rcvd: 53

✅ Expected A-record is returned

Thanks for looking into it this quickly!

rgacogne · 2022-06-29T10:00:14Z

Thanks a lot for testing and reporting back! I'll add the missing unit and regression tests to the pull request quickly, then after a review it will be merged into the development branch, and I'll then backport it to the 1.7.x branch.
I don't have a release date for the next 1.7.x release yet, so in the meantime you can either switch to the packages that will be automatically built once the backport PR has been merged, or use the version you compiled which should be mostly identical.

KlettIT · 2022-07-01T15:51:32Z

Thanks for the "next steps"-explanation!
As we added your commit to rel/dnsdist-1.7.x branch I think we will stay on our "custom build" until the next 1.7.x gets released.

I am not sure if this is even supposed to work but I found that SetEDNSOptionAction() seems to be only working directly used in dnsdist.conf.
If I call it from within a LUA-function it seems to just do nothing...?

Should I open another issue or is this expected?

Sample configuration

setLocal("127.0.0.20:53")
newServer({address="208.67.222.222", name="resolver1.opendns.com", order=1, pool="pool", reconnectOnUp=true, maxCheckFailures=3, rise=3})

function AddEDNS(dq)
  SetEDNSOptionAction(20292,"test")
  return DNSAction.Pool, "pool"
end

ModifiedInMain = newSuffixMatchNode()
ModifiedInLua = newSuffixMatchNode()

ModifiedInMain:add(newDNSName("foo.tld."))
ModifiedInLua:add(newDNSName("bar.tld."))

addAction(SuffixMatchNodeRule(ModifiedInMain), SetEDNSOptionAction(20292,"test"))
addAction(SuffixMatchNodeRule(ModifiedInLua), LuaAction(AddEDNS))

addAction(AllRule(), PoolAction("pool"))

Query foo.tld

dig @127.0.0.20 foo.tld A

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.9 <<>> @127.0.0.20 foo.tld A
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 23924
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;foo.tld.                       IN      A

;; AUTHORITY SECTION:
.                       3600    IN      SOA     a.root-servers.net. nstld.verisign-grs.com. 2022070100 1800 900 604800 86400

;; Query time: 4 msec
;; SERVER: 127.0.0.20#53(127.0.0.20)
;; WHEN: Fri Jul 01 17:01:43 CEST 2022
;; MSG SIZE  rcvd: 111

✅ EDNS option gets added

17:01:43.898442 IP (tos 0x0, ttl 64, id 7314, offset 0, flags [none], proto UDP (17), length 64)
    127.0.0.1.43751 > 127.0.0.20.53: [bad udp cksum 0xfe52 -> 0x02f7!] 23924+ [1au] A? foo.tld. ar: . OPT UDPsize=4096 (36)
        0x0000:  0000 0304 0006 0000 0000 0000 0000 0800  ................
        0x0010:  4500 0040 1c92 0000 4011 6006 7f00 0001  E..@....@.`.....
        0x0020:  7f00 0014 aae7 0035 002c fe52 5d74 0120  .......5.,.R]t..
        0x0030:  0001 0000 0000 0001 0366 6f6f 0374 6c64  .........foo.tld
        0x0040:  0000 0100 0100 0029 1000 0000 0000 0000  .......)........
17:01:43.898643 IP (tos 0x0, ttl 64, id 35131, offset 0, flags [DF], proto UDP (17), length 72)
    10.195.129.37.39946 > 208.67.222.222.53: [bad udp cksum 0x3b50 -> 0xfa18!] 256+ [1au] A? foo.tld. ar: . OPT UDPsize=4096 (44)
        0x0000:  0004 0001 0006 001a 4a16 032f 0000 0800  ........J../....
        0x0010:  4500 0048 893b 4000 4011 765f 0ac3 8125  E..H.;@.@.v_...%
        0x0020:  d043 dede 9c0a 0035 0034 3b50 0100 0120  .C.....5.4;P....
        0x0030:  0001 0000 0000 0001 0366 6f6f 0374 6c64  .........foo.tld
        0x0040:  0000 0100 0100 0029 1000 0000 0000 0008  .......)........
        0x0050:  4f44 0004 7465 7374                      OD..test

Query bar.tld

dig @127.0.0.20 bar.tld A

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.9 <<>> @127.0.0.20 bar.tld A
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 58934
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;bar.tld.                       IN      A

;; AUTHORITY SECTION:
.                       3600    IN      SOA     a.root-servers.net. nstld.verisign-grs.com. 2022070100 1800 900 604800 86400

;; Query time: 5 msec
;; SERVER: 127.0.0.20#53(127.0.0.20)
;; WHEN: Fri Jul 01 17:03:19 CEST 2022
;; MSG SIZE  rcvd: 111

❌ No EDNS option is added

17:03:19.065238 IP (tos 0x0, ttl 64, id 51582, offset 0, flags [none], proto UDP (17), length 64)
    127.0.0.1.37000 > 127.0.0.20.53: [bad udp cksum 0xfe52 -> 0xa294!] 58934+ [1au] A? bar.tld. ar: . OPT UDPsize=4096 (36)
        0x0000:  0000 0304 0006 0000 0000 0000 0000 0800  ................
        0x0010:  4500 0040 c97e 0000 4011 b319 7f00 0001  E..@.~..@.......
        0x0020:  7f00 0014 9088 0035 002c fe52 e636 0120  .......5.,.R.6..
        0x0030:  0001 0000 0000 0001 0362 6172 0374 6c64  .........bar.tld
        0x0040:  0000 0100 0100 0029 1000 0000 0000 0000  .......)........
17:03:19.065354 IP (tos 0x0, ttl 64, id 19566, offset 0, flags [DF], proto UDP (17), length 64)
    10.195.129.37.39946 > 208.67.222.222.53: [bad udp cksum 0x3b48 -> 0x3d54!] 768+ [1au] A? bar.tld. ar: . OPT UDPsize=4096 (36)
        0x0000:  0004 0001 0006 001a 4a16 032f 0000 0800  ........J../....
        0x0010:  4500 0040 4c6e 4000 4011 b334 0ac3 8125  E..@Ln@.@..4...%
        0x0020:  d043 dede 9c0a 0035 002c 3b48 0300 0120  .C.....5.,;H....
        0x0030:  0001 0000 0000 0001 0362 6172 0374 6c64  .........bar.tld
        0x0040:  0000 0100 0100 0029 1000 0000 0000 0000  .......)........

rgacogne · 2022-07-01T19:58:44Z

SetEDNSOptionAction is an action, it can only be used in conjunction with addAction, not in custom Lua actions. If you want to add custom EDNS options via a Lua action you would need a different binding, which doesn't seem to exist today. I would suggest opening a feature request, or even better a pull request if you feel up to writing that code.

rgacogne added defect dnsdist labels Jun 28, 2022

rgacogne added this to the dnsdist-1.8.0 milestone Jun 28, 2022

rgacogne mentioned this issue Jun 28, 2022

dnsdist: Fix a bug in SetEDNSOptionAction #11729

Merged

7 tasks

rgacogne closed this as completed in #11729 Oct 6, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SetEDNSOption does not work as expected with dnssec enabled #11728

SetEDNSOption does not work as expected with dnssec enabled #11728

KlettIT commented Jun 28, 2022 •

edited

Loading

Habbie commented Jun 28, 2022

Habbie commented Jun 28, 2022

KlettIT commented Jun 28, 2022 •

edited

Loading

KlettIT commented Jun 28, 2022

rgacogne commented Jun 28, 2022

rgacogne commented Jun 28, 2022

KlettIT commented Jun 28, 2022

KlettIT commented Jun 28, 2022

rgacogne commented Jun 29, 2022

KlettIT commented Jul 1, 2022

rgacogne commented Jul 1, 2022 •

edited

Loading

SetEDNSOption does not work as expected with dnssec enabled #11728

SetEDNSOption does not work as expected with dnssec enabled #11728

Comments

KlettIT commented Jun 28, 2022 • edited Loading

Short description

Environment

Sample configuration

Steps to reproduce

1.) Query test.com without dnssec

2.) Query test.com with dnssec

Verification

1.) Query some other domain without dnssec

2.) Query the same domain with dnssec

Expected behaviour

Actual behaviour

Habbie commented Jun 28, 2022

Habbie commented Jun 28, 2022

KlettIT commented Jun 28, 2022 • edited Loading

KlettIT commented Jun 28, 2022

rgacogne commented Jun 28, 2022

rgacogne commented Jun 28, 2022

KlettIT commented Jun 28, 2022

KlettIT commented Jun 28, 2022

rgacogne commented Jun 29, 2022

KlettIT commented Jul 1, 2022

Sample configuration

Query foo.tld

Query bar.tld

rgacogne commented Jul 1, 2022 • edited Loading

KlettIT commented Jun 28, 2022 •

edited

Loading

1.) Query `test.com` without dnssec

2.) Query `test.com` with dnssec

KlettIT commented Jun 28, 2022 •

edited

Loading

rgacogne commented Jul 1, 2022 •

edited

Loading