Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DNS Blacklists for threat shield #6212

Closed
cotosso opened this issue Jun 24, 2020 · 15 comments
Closed

DNS Blacklists for threat shield #6212

cotosso opened this issue Jun 24, 2020 · 15 comments
Labels
verified All test cases were verified successfully

Comments

@cotosso
Copy link

cotosso commented Jun 24, 2020 •
edited by edospadoni
Loading

DNS blacklists would be a nice addition for threat shield module, cause they can be very effective and provide a greater protection for clients over the IP blacklists.

Proposed solution

  • I propose to create a new DNS proxy that send requests to DNS that provide this kind of service
  • it should have the possibility to set the DNS proxy as transparent and allow to choose zones were the service must be active.
  • it should have bypass options for sources based on firewall objects
  • It could have a new settings page for threath shield and we could have ip blacklists settings (the page we have now) and dns blacklists settings in 2 separate pages.

Steps

  • UI. Create new DNS Blacklist page on Cockpit, under Threat shield module (Dashboard and DNS Proxy page)
  • Backend. Add new DNS proxy lists and configure them using pi-hole, retrieve statistics and implement bypass in shorewall
@gsanchietti gsanchietti mentioned this issue Jun 24, 2020
Merged
@andre8244 andre8244 mentioned this issue Jun 24, 2020
Merged
@andre8244 andre8244 mentioned this issue Jun 26, 2020
Merged
edospadoni pushed a commit to NethServer/nethserver-blacklist that referenced this issue Jun 26, 2020
@andre8244
Dns proxy UI (#18)
7631d9e 
* api & ui: IP and DNS settings and analisys

NethServer/dev#6212
This was referenced Jun 26, 2020
Merged
Merged
Merged
gsanchietti pushed a commit to nethesis/nethserver-flashstart that referenced this issue Jun 29, 2020
@andre8244
api & ui: show warning if ftl is enabled (#9)
ffbe47c 
FlashStart cannot operate while ftl (DNS blacklist) is enabled

NethServer/dev#6212
gsanchietti pushed a commit to NethServer/docs that referenced this issue Jun 29, 2020
@andre8244 @filippocarletti
Threat shield: DNS blacklist (#512)
21efbfb 
DNS blacklist documentation for Threat shield

NethServer/dev#6212

Co-authored-by: Filippo Carletti <filippo.carletti@gmail.com>
@andre8244 andre8244 mentioned this issue Jun 29, 2020
Merged
gsanchietti added a commit to NethServer/nethserver-blacklist that referenced this issue Jun 29, 2020
@gsanchietti @edospadoni @andre8244
Dns proxy (#17)
f811b31 
- gitignore: remove rpm and tar.gz
- spec: require pihole-ftl
- templates and events: implement DNS filter
- cockpit: implement UI for DNS filter

NethServer/dev#6212

Co-authored-by: Giacomo Sanchietti <giacomo.sanchietti@nethesis.it>
Co-authored-by: Edoardo Spadoni <edoardo.spadoni@nethesis.it>
Co-authored-by: Andrea Leardini <andre8244@gmail.com>
@nethbot
Copy link
Member

nethbot commented Jun 29, 2020

in 7.8.2003/testing:

edospadoni pushed a commit to NethServer/nethserver-blacklist that referenced this issue Jun 30, 2020
@andre8244
doc. added DNS blacklist to README (#19)
3ea8fdb 
NethServer/dev#6212
@gsanchietti gsanchietti mentioned this issue Jun 30, 2020
Merged
@nethbot
Copy link
Member

nethbot commented Jun 30, 2020

in 7.8.2003/testing:

@nethbot
Copy link
Member

nethbot commented Jun 30, 2020

in 7.8.2003/testing:

@gsanchietti gsanchietti mentioned this issue Jun 30, 2020
Merged
edospadoni added a commit to NethServer/docs that referenced this issue Jun 30, 2020
@edospadoni
docs. update blacklist chapter
f6ec8e4 
NethServer/dev#6212
@gsanchietti
Copy link
Member

Test case 1

  • Install on a clean machine or upgrade an existing one
  • Access the new DNS blacklist page inside Threat shield application
  • Enable the service and set https://github.com/NethServer/dns-community-blacklist.git as download URL
  • Enable the category
  • From a client in the green zone, try to query a blocked domain on the server using port 1153. Example: dig -p 1153 @your_firewall test.org

Test case 2

  • After test case 1
  • Make sure the client can still execute queries to an external DNS server

Test case 3

  • After test case 2, enable the DNS proxy on green zone
  • Verify the client on green zone can't execute queries to an external DNS server: all requests must be redirect to the local DNS proxy

Test case 4

  • After test case 3
  • Add the host to the bypass list
  • Make sure the client can now execute queries to an external DNS server

@gsanchietti gsanchietti added the testing Packages are available from testing repositories label Jun 30, 2020
@edospadoni edospadoni mentioned this issue Jun 30, 2020
Merged
gsanchietti added a commit to NethServer/docs that referenced this issue Jun 30, 2020
@gsanchietti
blacklist: add sample DNS blacklist (#515)
736cde9 
Users will have a sample repository to test the DNS blacklist.

NethServer/dev#6212
@mamengoni mamengoni self-assigned this Jul 1, 2020
@mamengoni
Copy link

Test case 1
Tested and verified!

Test case 2
Tested and verified

Test case 3
Tested and verified

Test case 4
Tested and verified

@mamengoni mamengoni added verified All test cases were verified successfully and removed testing Packages are available from testing repositories labels Jul 1, 2020
@nethbot
Copy link
Member

nethbot commented Jul 2, 2020

in 7.8.2003/updates:

@gsanchietti gsanchietti reopened this Jul 2, 2020
@gsanchietti
Copy link
Member

gsanchietti commented Jul 2, 2020 •
edited
Loading

Back to testing since pihole crashed under not so heavy load, the problem needs further investigation.

Reported error:

[2020-07-02 17:00:04.763 2732] Received: Real-time signal 0 (34 -> 0)
[2020-07-02 17:00:04.764 2732] SQLite3 message: file unlinked while open: /etc/pihole/gravity.db (28)
[2020-07-02 17:00:06.393 2732] INFO: No regex blacklist entries found
[2020-07-02 17:00:06.393 2732] INFO: No regex whitelist entries found
[2020-07-02 17:00:06.407 2732] Compiled 0 whitelist and 0 blacklist regex filters in 14.3 msec
[2020-07-02 17:00:08.031 2732] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-07-02 17:00:08.031 2732] ---------------------------->  FTL crashed!  <----------------------------
[2020-07-02 17:00:08.031 2732] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-07-02 17:00:08.031 2732] Please report a bug at https://github.com/pi-hole/FTL/issues
[2020-07-02 17:00:08.031 2732] and include in your report already the following details:
[2020-07-02 17:00:08.031 2732] FTL has been running for 3791 seconds
[2020-07-02 17:00:08.031 2732] FTL branch: master
[2020-07-02 17:00:08.031 2732] FTL version: v5.0
[2020-07-02 17:00:08.031 2732] FTL commit: 3d7c095
[2020-07-02 17:00:08.031 2732] FTL date: 2020-05-10 18:58:38 +0100
[2020-07-02 17:00:08.031 2732] FTL user: started as root, ended as root
[2020-07-02 17:00:08.031 2732] Compiled for x86_64 (compiled on CI) using gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
[2020-07-02 17:00:08.031 2732] Received signal: Segmentation fault
[2020-07-02 17:00:08.031 2732]      at address: 0x30
[2020-07-02 17:00:08.031 2732]      with code: SEGV_MAPERR (Address not mapped to object)
[2020-07-02 17:00:08.031 2732] Backtrace:
[2020-07-02 17:00:08.031 2732] B[0000]: 0x5644b31846f9, /usr/bin/pihole-ftl(+0x316f9) [0x5644b31846f9]
[2020-07-02 17:00:08.031 2732] B[0001]: 0x7f4df58d1630, /lib64/libpthread.so.0(+0xf630) [0x7f4df58d1630]
[2020-07-02 17:00:08.031 2732] B[0002]: 0x5644b32766a1, /usr/bin/pihole-ftl(+0x1236a1) [0x5644b32766a1]
[2020-07-02 17:00:08.031 2732] B[0003]: 0x5644b3283c68, /usr/bin/pihole-ftl(+0x130c68) [0x5644b3283c68]
[2020-07-02 17:00:08.031 2732] B[0004]: 0x5644b317bbe8, /usr/bin/pihole-ftl(+0x28be8) [0x5644b317bbe8]
[2020-07-02 17:00:08.031 2732] B[0005]: 0x5644b317d392, /usr/bin/pihole-ftl(in_gravity+0x112) [0x5644b317d392]
[2020-07-02 17:00:08.031 2732] B[0006]: 0x5644b318803e, /usr/bin/pihole-ftl(+0x3503e) [0x5644b318803e]
[2020-07-02 17:00:08.031 2732] B[0007]: 0x5644b31894d7, /usr/bin/pihole-ftl(_FTL_new_query+0x707) [0x5644b31894d7]
[2020-07-02 17:00:08.031 2732] B[0008]: 0x5644b31a31ad, /usr/bin/pihole-ftl(receive_query+0xa6d) [0x5644b31a31ad]
[2020-07-02 17:00:08.031 2732] B[0009]: 0x5644b31b994b, /usr/bin/pihole-ftl(+0x6694b) [0x5644b31b994b]
[2020-07-02 17:00:08.031 2732] B[0010]: 0x5644b31bb7bc, /usr/bin/pihole-ftl(main_dnsmasq+0x129c) [0x5644b31bb7bc]
[2020-07-02 17:00:08.031 2732] B[0011]: 0x5644b31778ac, /usr/bin/pihole-ftl(main+0xdc) [0x5644b31778ac]
[2020-07-02 17:00:08.031 2732] B[0012]: 0x7f4df5516555, /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f4df5516555]
[2020-07-02 17:00:08.031 2732] B[0013]: 0x5644b3177a1a, /usr/bin/pihole-ftl(_start+0x2a) [0x5644b3177a1a]

This seems related to pi-hole/FTL#816

@gsanchietti gsanchietti mentioned this issue Jul 2, 2020
Merged
gsanchietti added a commit to NethServer/nethserver-blacklist that referenced this issue Jul 2, 2020
@gsanchietti
dns: just reload ftl (#24)
e16a82d 
Do not restart ftl to avoid service interruption.
The service can be reloaded using the following signals:
- SIGRTMIN: just reload the lists
- SIGHUP: flush DNS cache

Both signals will NOT re-read any *.conf files.

NethServer/dev#6212
@gsanchietti gsanchietti mentioned this issue Jul 2, 2020
Merged
@nethbot
Copy link
Member

nethbot commented Jul 2, 2020

in 7.8.2003/testing:

gsanchietti added a commit to NethServer/pihole-ftl that referenced this issue Jul 2, 2020
@gsanchietti
systemd: restart on failure (#1)
49e4722 
When used as DNS proxy, ftl should be restarted by systemd if a crash
occurs.

NethServer/dev#6212
@gsanchietti gsanchietti removed the verified All test cases were verified successfully label Jul 2, 2020
@gsanchietti
Copy link
Member

In testing:

  • pihole-ftl-5.0-2.ns7.x86_64.rpm

@gsanchietti
Copy link
Member

Back to testing, let's see how it perform on a real environment

@gsanchietti gsanchietti added the testing Packages are available from testing repositories label Jul 2, 2020
@edospadoni edospadoni mentioned this issue Jul 3, 2020
Merged
edospadoni added a commit to NethServer/nethserver-blacklist that referenced this issue Jul 3, 2020
@edospadoni @gsanchietti
cron. move download dns once per day (#25)
abaa2fa 
NethServer/dev#6212

Co-authored-by: Giacomo Sanchietti <giacomo.sanchietti@nethesis.it>
@nethbot
Copy link
Member

nethbot commented Jul 3, 2020

in 7.8.2003/testing:

@edospadoni edospadoni mentioned this issue Jul 3, 2020
Merged
gsanchietti pushed a commit to NethServer/pihole-ftl that referenced this issue Jul 3, 2020
@edospadoni @gsanchietti
pihole conf. added max db days and log age (#2)
df46832 
When pihole-FTL.db contains many records the restart of ftl service takes long time and causing disruptions. To reduce restart time we add these configurations:

- MAXLOGAGE to 1 hour
- MAXDBDAYS to 7 days

NethServer/dev#6212
@gsanchietti
Copy link
Member

In testing:
pihole-ftl-5.0-3.ns7.x86_64.rpm

@gsanchietti gsanchietti self-assigned this Jul 6, 2020
andre8244 added a commit to andre8244/nethserver-blacklist that referenced this issue Jul 6, 2020
@andre8244
ui: human numbers for DNS blacklist
7d38158 
NethServer/dev#6212
@andre8244 andre8244 mentioned this issue Jul 6, 2020
Merged
edospadoni pushed a commit to NethServer/nethserver-blacklist that referenced this issue Jul 7, 2020
@andre8244
ui: human numbers for DNS blacklist (#26)
8b85b90 
NethServer/dev#6212
@nethbot
Copy link
Member

nethbot commented Jul 7, 2020

in 7.8.2003/testing:

@gsanchietti
Copy link
Member

No more crashed found, also the restart now takes about only 1 second.

Verified.

@gsanchietti gsanchietti removed their assignment Jul 7, 2020
@gsanchietti gsanchietti added verified All test cases were verified successfully and removed testing Packages are available from testing repositories labels Jul 7, 2020
@nethbot
Copy link
Member

nethbot commented Jul 7, 2020 •
edited by gsanchietti
Loading

in 7.8.2003/updates:

  • nethserver-blacklist-1.1.1-1.ns7.noarch.rpm x86_64 armhfp aarch64
  • pihole-ftl-5.0-3.ns7.x86_64.rpm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
verified All test cases were verified successfully
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@gsanchietti @andre8244 @edospadoni @cotosso @mamengoni @nethbot