Unbound - Blocklist.site URLs No Longer Working - OPNsense Unable to Download DNSBL Lists from Blocklist.site

[daygle]

Important notices

Before you add a new report, we ask you kindly to acknowledge the following:

• I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
• I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Describe the bug

Just wanted to report that it appears that Blocklist.site lists appear to have changed website/URL for the DNSBL lists. OPNsense now reporting 'Failed to establish a new connection: [Errno 8] Name does not resolve' when attempting to download DNSBL lists from Blocklist.site.

To Reproduce

Steps to reproduce the behavior:

1. Go to 'Unbound DNS'
2. Click on 'Log File'
3. See errors (screenshot attached).

Expected behavior

OPNsense/Unbound should correctly download Blocklist.site DNSBL.

Describe alternatives you considered

As a workaround I have added the below DNSBL to the 'URLs of Blocklists' within Unbound.

https://blocklistproject.github.io/Lists/alt-version/abuse-nl.txt
https://blocklistproject.github.io/Lists/alt-version/drugs-nl.txt
https://blocklistproject.github.io/Lists/alt-version/fraud-nl.txt
https://blocklistproject.github.io/Lists/alt-version/malware-nl.txt
https://blocklistproject.github.io/Lists/alt-version/phishing-nl.txt
https://blocklistproject.github.io/Lists/alt-version/ransomware-nl.txt
https://blocklistproject.github.io/Lists/alt-version/scam-nl.txt
https://blocklistproject.github.io/Lists/alt-version/tracking-nl.txt
https://blocklistproject.github.io/Lists/alt-version/porn-nl.txt

Screenshots

Relevant log files

Log file shown above in screenshot.

Additional context

N/A

Environment

Software version used and hardware type if relevant, e.g.:

OPNsense 23.1.6-amd64
FreeBSD 13.1-RELEASE-p7
OpenSSL 1.1.1t 7 Feb 2023

[daygle]

Sorry - closing. Must of been a once off issue.

[Chaskel]

Hi @fichtner , I am wondering if the issue described above is what I recently commented on in the following post?

https://forum.opnsense.org/index.php?topic=32052.msg162636#msg162636

If so, should I open up a new issue, or would you like to use this existing one?

Thank you.

[AdSchellevis]

if it's about Failed to establish a new connection: [Errno 8] Name does not resolve, I would say the message is quite self-explanatory, name server not (or not correct) configured on the firewall itself.

[Chaskel]

Thank you @AdSchellevis for your quick response. It is indeed quite possible it is DNS-related, but not necessarily due to upstream DNS server issue. I am going to post what I mentioned in the forum here in case it helps (as I have some thoughts on what could possibly be going on listed):

"I too have experienced this issue in more recent versions. Unfortunately I am unable to say when I started noticing the change, but here is some information in case it helps determine what could be going on...

1. Reboot of OPNSense at 2 locations I have running OPNsense 23.1.5_4-amd64 yields the following each time:

Notice unbound blocklist: https://adaway.org/hosts.txt (exclude: 0 block: 0)
Notice unbound blocklist download: 0 total lines downloaded for https://adaway.org/hosts.txt
Error unbound blocklist download : unable to download file from https://adaway.org/hosts.txt (error : HTTPSConnectionPool(host='adaway.org', port=443): Max retries exceeded with url: /hosts.txt (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x8027cf640>: Failed to establish a new connection: [Errno 8] Name does not resolve')))

2. Manual restarting of Unbound service (e.g. restart service button on Blocklist page) does not appear to initiate download of list (based on not seeing messages such as those listed above).

3. If I disable Blocklist/Apply, then Enable Blocklist/Apply, it appears to trigger getting data:

Notice unbound blocklist parsing done in 0.58 seconds (7355 records)
Notice unbound blocklist: https://adaway.org/hosts.txt (exclude: 2 block: 7355)
Notice unbound blocklist download: 11782 total lines downloaded for https://adaway.org/hosts.txt
Notice unbound blocklist download : exclude domains matching ^(?![a-zA-Z_\d]).*|.*localhost$

NOTE: Even though the data seems to be retrieved, it appears it is not active until I then restart the service* (e.g. restart service button on Blocklist page).

*It also seems as though I need to go through the disable/enable steps then restart service an additional time to have everything fully work. I am not sure if it is always just one time, but I do know that doing the entire process once does not usually get everything working.

DNS config information that may be of interest:

1. Services->Unbound DNS->Blocklist - "AdAway List" selected and all other fields empty.
2. Services->Unbound DNS->DNS over TLS - 2 IPv4 and 2 IPv6 servers defined. All 4 using port 853.
3. Services->Unbound DNS->General - DNSSEC support enabled.
4. System->Settings->General - No DNS servers manually defined.
5. System->Settings->General - Allow DNS server list to be overridden by DHCP/PPP on WAN is enabled.

If it is not a setting issue, I am wondering if perhaps the following may relate to what I am seeing:

1. For bootup situation (DNS resolution error), perhaps a service dependency needs to be made if the blocklist process is launching before DNS resolution services are fully up and running (if that is what is actually happening).

2. For the manual service restart item, perhaps there are additional processes that need to be restarted behind the scenes as part of the service restart to trigger getting the URL to process the data.

I hope the above is helpful."

Thank you

[AdSchellevis]

@Chaskel likely a problem with settings, best investigate local dns access first, check if other hosts are accessible. In case anything goes wrong on Unbounds end, I would expect you can find more information in it's log.

[Chaskel]

Thank you @AdSchellevis . Both the unsuccessful (system boot) and successful (manually initiated post-boot) log entries I referenced in my last post are actually from the Unbound log file. In both of my OPNsense locations I see local DNS from clients work as well as from as the OPNsense Interfaces: Diagnostics: DNS Lookup tool.

As the issue appears to happen only at boot each time at both of my OPNsense locations, that is what makes me wonder if a dependent process may not be fully starting before Unbound blocklist-related processes are starting up. Is there any possibility that in a configuration like I have where Unbound DNS over TLS is used (or maybe even DNSSEC that I reference having enabled), that those name resolution services are not fully up and running yet when the blocklist-related processes starts up?

Thank you

[AdSchellevis]

@Chaskel so dns works on boot, but only not from the local machine at that time? The resolve error is only recorded after boot and a manual download works without issues? (you can trigger this using configctl unbound dnsbl)

[Chaskel]

@AdSchellevis as far as I am aware, I am not having any issues with clients relating to DNS. What made me realize something was not working correctly is when web browsing clients started getting advertisements (which led me to check out the Unbound logs and discover the name resolution issue).

DNS appears to work for everything except for the Unbound blocklist download process at boot, and yes manual download works, however please note the specifics I previously referenced that after manual download (which I have been using the GUI steps I mentioned), I have to do several steps (and some more than once), to have the actual blocklist start working for the web browsing clients.

Thank you

[AdSchellevis]

@Chaskel can you remove /usr/local/etc/rc.syshook.d/start/85-dnsbl and reboot? If that fixes it, an issue might be practical indeed.

[Chaskel]

Sure @AdSchellevis I can do that. I will need to do a little later as in the last few minutes I had to put a whitelist domain entry in for my family to be able to do some work-related items. I will provide you with an update once I am able to reboot.

Do you think it is worth you reopening this Github issue for now?

[AdSchellevis]

@Chaskel if this is the issue, better open a new ticket. I expect it would be better if the previous downloaded data will be used on boot, but that's something we can discuss when there's an issue with a clear cause.

[Chaskel]

@AdSchellevis I removed the file. I'm listing what I did in case the order, or method used could change outcome:

1. Enabled root user via web GUI (so that I could remove the file).
2. SSH'd to OPNsense system, went to shell, and removed the file.
3. I believe I then did a sudo reboot (I am not sure if I should have done that as it is habit from Linux devices I work with)
4. Once device was back up I disabled root user via the web GUI.
5. I looked for instances in Unbound log of blocklist trying to download files and saw none (I suspect this is expect based on removing the file so that /usr/local/sbin/configctl -dq unbound dnsbl does not run on boot)
6. On Windows client I ran ipconfig /flushdns
7. I then pulled up some web sites and confirmed no advertisements in web browser.

Would you like me to do anything else, and if not, should I create 2 issues (one for the boot issue and another for the manual load issue - and what what you recommend I title the issue(s))?

@daygle , in case you want to see if the problem you created this particular Github issue for does show back up if you reboot, feel free to share.

Thank you

[daygle]

Oh wow! I woke up this morning and many emails with messages for this issue - wasn't expecting this many replies for my closed case.

@AdSchellevis So are you saying you think that I will receive the error again after a reboot of OPNsense?

[daygle]

@AdSchellevis I suspect that my issue was caused because I made the below change, but not sure?

https://homenetworkguy.com/how-to/redirect-all-dns-requests-to-local-dns-resolver/

[Chaskel]

Hi @daygle , it is more of a question of curiosity on my part to see if you see the same behavior on reboot.

[Chaskel]

@AdSchellevis in addition to my last questions for you on if you would like me to try anything else as well as what your recommendation is for creating 1 or 2 new Github issues (and what to title them for you), I wanted to provide the following update.

Just now I reverted the change you had me do earlier today (remove file) and I observed a message I thought I had seen once before prior to me escalating this issue (but I did not have a copy of the exact message from the first time I saw it). I am not sure why this particular message sometimes randomly appears during restart instead of the usual "[Errno 8] Name does not resolve" we have been discussing:

2023-04-23T19:06:35-07:00 | Notice | unbound | blocklist: https://adaway.org/hosts.txt (exclude: 0 block: 0) |  
2023-04-23T19:06:35-07:00 | Notice | unbound | blocklist download: 0 total lines downloaded for https://adaway.org/hosts.txt |
2023-04-23T19:06:35-07:00 | Error | unbound | blocklist download : unable to download file from https://adaway.org/hosts.txt (error : HTTPSConnectionPool(host='adaway.org', port=443): Max retries exceeded with url: /hosts.txt (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x8027cf640>: Failed to establish a new connection: [Errno 13] Permission denied')))

Current listing for the directory:
drwxr-xr-x 2 root wheel 10 Apr 23 19:03 .
drwxr-xr-x 9 root wheel 9 Feb 21 2022 ..
-rwxr-xr-x 1 root wheel 921 Mar 30 23:06 10-newwanip
-rwxr-xr-x 1 root wheel 78 Mar 30 23:06 20-freebsd
-rwxr-xr-x 1 root wheel 197 Mar 30 23:06 25-syslog
-rwxr-xr-x 1 root wheel 55 Apr 23 18:56 85-dnsbl
-rwxr-xr-x 1 root wheel 1539 Mar 30 23:06 90-carp
-rwxr-xr-x 1 root wheel 106 Mar 30 23:06 90-cron
-rwxr-xr-x 1 root wheel 121 Mar 30 23:06 90-sysctl
-rwxr-xr-x 1 root wheel 31 Mar 30 23:06 95-beep

Contents of 85-dnsbl:
#!/bin/sh

/usr/local/sbin/configctl -dq unbound dnsbl

Just like the one time I saw that error before, a following reboot did not show the same:

2023-04-23T19:21:34-07:00 | Notice | unbound | blocklist: https://adaway.org/hosts.txt (exclude: 0 block: 0) |  
2023-04-23T19:21:34-07:00 | Notice | unbound | blocklist download: 0 total lines downloaded for https://adaway.org/hosts.txt |
2023-04-23T19:21:34-07:00 | Error | unbound | blocklist download : unable to download file from https://adaway.org/hosts.txt (error : HTTPSConnectionPool(host='adaway.org', port=443): Max retries exceeded with url: /hosts.txt (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x8027ce640>: Failed to establish a new connection: [Errno 8] Name does not resolve')))

Thank you, and I hope this additional information is possibly helpful.

[AdSchellevis]

If removing the file makes sure the name resolve doesn't happen on boot, just open a ticket for that. I can think of multiple reasons why other configuration issues may lead to not being able to access the resource, but that really lies outside of community support scope. Let's try to keep this simple and focused to the issue at hand, my time is rather limited.

[Chaskel]

Hi @AdSchellevis , I greatly appreciate your time and support. Issue 6523 has now been created for this.