Response change to NODATA for some ANY queries since 1.12, tested on 1.16.1

[jdamick]

Describe the bug

Behavior change in the response from certain TLDs noticed between 1.12 and 1.16.1 when querying ".br" with ANY for example it now returns NODATA. The SOA is moved to the authority section from the answer. It appears that unbound is 'fixing' the answer and moving the soa to the authority section and then caching it. This is a divergence in response behavior and doesn't match other providers, for example: https://dns.google/resolve?name=br&type=ANY&do=true or

To reproduce
Steps to reproduce the behavior:

1. Start with Empty cache & dnssec validation enabled
2. dig br. -t A
3. dig br. -t ANY

Expected behavior
A clear and concise description of what you expected to happen.

unbound 1.12

; <<>> DiG xxxxx <<>> @x.x.x.x -p xxxx br. -t ANY
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22656
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;br.                IN  ANY

;; ANSWER SECTION:
br.         86400   IN  SOA a.dns.br. hostmaster.registro.br. 2023005472 1800 900 604800 900
br.         86400   IN  RRSIG   SOA 13 1 172800 20230119194009 20230105184009 32863 br. WNpA15lKNCOtTbXr0D2wzgFmzzaaGiymBg5jZwnZ0Q4zDAoCHzWCEBis UY/KLs9z8AdBj3+zTSyA4cwSCdMOWw==

;; Query time: 235 msec
;; SERVER: x.x.x.x#xxx (x.x.x.x)
;; WHEN: Thu Jan  5 20:09:43 2023
;; MSG SIZE  rcvd: 180

unbound 1.16.1

; <<>> DiG xxxxx <<>> @x.x.x.x -p xxxx br. -t ANY
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62406
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;br.                IN  ANY

;; AUTHORITY SECTION:
br.         897 IN  SOA a.dns.br. hostmaster.registro.br. 2023005482 1800 900 604800 900

;; Query time: 0 msec
;; SERVER: x.x.x.x#xxx (x.x.x.x)
;; WHEN: Thu Jan  5 20:08:57 2023
;; MSG SIZE  rcvd: 82

logs from 1.16.1

Jan 05 16:56:05 unbound[17461:16] debug: iter_handle processing q with state QUERY RESPONSE STATE
Jan 05 16:56:05 unbound[17461:16] info: query response was ANSWER
Jan 05 16:56:05 unbound[17461:16] debug: iter_handle processing q with state FINISHED RESPONSE STATE
Jan 05 16:56:05 unbound[17461:16] info: finishing processing for br. DNSKEY IN
Jan 05 16:56:05 unbound[17461:16] debug: mesh_run: iterator module exit state is module_finished
Jan 05 16:56:05 unbound[17461:16] debug: validator[module 0] operate: extstate:module_wait_module event:module_event_moddone
Jan 05 16:56:05 unbound[17461:16] info: validator operate: query br. DNSKEY IN
Jan 05 16:56:05 unbound[17461:16] debug: validator: nextmodule returned
Jan 05 16:56:05 unbound[17461:16] debug: not validating response, is valrec(validation recursion lookup)
Jan 05 16:56:05 unbound[17461:16] debug: mesh_run: validator module exit state is module_finished
Jan 05 16:56:05 unbound[17461:16] info: validator: inform_super, sub is br. DNSKEY IN
Jan 05 16:56:05 unbound[17461:16] info: super is br. A IN
Jan 05 16:56:05 unbound[17461:16] debug: attempt DS match algo 13 keytag 2471
Jan 05 16:56:05 unbound[17461:16] debug: DS match digest ok, trying signature
Jan 05 16:56:05 unbound[17461:16] debug: DS matched DNSKEY.
Jan 05 16:56:05 unbound[17461:16] info: validated DNSKEY br. DNSKEY IN
Jan 05 16:56:05 unbound[17461:16] debug: validator[module 0] operate: extstate:module_wait_subquery event:module_event_pass
Jan 05 16:56:05 unbound[17461:16] info: validator operate: query br. A IN
Jan 05 16:56:05 unbound[17461:16] debug: val handle processing q with state VAL_FINDKEY_STATE
Jan 05 16:56:05 unbound[17461:16] info: validator: FindKey br. A IN
Jan 05 16:56:05 unbound[17461:16] debug: val handle processing q with state VAL_VALIDATE_STATE
Jan 05 16:56:05 unbound[17461:16] info: verify rrset br. SOA IN
Jan 05 16:56:05 unbound[17461:16] debug: verify sig 32863 13
Jan 05 16:56:05 unbound[17461:16] debug: verify result: sec_status_secure
Jan 05 16:56:05 unbound[17461:16] info: verify rrset br. NSEC IN
Jan 05 16:56:05 unbound[17461:16] debug: verify sig 32863 13
Jan 05 16:56:05 unbound[17461:16] debug: verify result: sec_status_secure
Jan 05 16:56:05 unbound[17461:16] debug: Validating a nodata response
Jan 05 16:56:05 unbound[17461:16] debug: successfully validated NODATA response.
Jan 05 16:56:05 unbound[17461:16] info: validate(nodata): sec_status_secure
Jan 05 16:56:05 unbound[17461:16] debug: val handle processing q with state VAL_FINISHED_STATE
Jan 05 16:56:05 unbound[17461:16] info: validation success br. A IN
Jan 05 16:56:05 unbound[17461:16] info: negcache insert for zone br. SOA IN
Jan 05 16:56:05 unbound[17461:16] info: negcache rr br. NSEC IN
Jan 05 16:56:05 unbound[17461:16] debug: mesh_run: validator module exit state is module_finished

System:

• Unbound version: 1.16.1
• OS: amzn-linux
• unbound -V output:

Version 1.16.1

Configure line: SWIG_LIB=<redacted>
Linked libs: libevent 2.1.11-stable (it uses epoll), OpenSSL 1.0.2u  20 Dec 2019
Linked modules: dns64 respip validator iterator

BSD licensed, see LICENSE in source package for details.
Report bugs to unbound-bugs@nlnetlabs.nl or https://github.com/NLnetLabs/unbound/issues

Additional information
Add any other information that you may have gathered about the issue here.

[wcawijngaards]

The issue seems to stem from the new default, that turns on aggressive-nsec. What was wrong was that the server used aggressive nsec information to disprove, with a nodata answer, the query of type ANY. The information is from the negative cache, built up with the type A query. The fix changes it so that type ANY queries do not receive nodata answers from the negative cache.

The default for this has been changed since 1.12, and this is why I guess the issue is now observable. Thanks for the report!