GoAccess for DNS query logs (Bind9)

[kpgreene]

Hello Group:
Has anyone considered the use of GoAccess to study DNS queries?
I get a reasonable amount of queries (~50/second) on my public DNS server and while it is simple to create logs, the ability to display in some sort of usable fashion is a challenge. I was thinking that GoAccess would be a good tool as it is fast and the ability to customize the log format is very useful. That said, I have tried for several hours to create the script to do it and I keep getting a fatal error in the startup process.

Any ideas where I am going wrong? Perhaps it is right in front of me and I do not really understand the file parsing. One line of a DNS query is shown.

This is my simple script with all my notes on it (useful when on a single screen). The spacing of the fields is OK when used on a bigger monitor.

Thank you.

Kevin

# Creates a view of traffic

# Fields needed                                         1       2               3       4       5       6       7       8     910      11      12      13
goaccess -f /var/log/named/queries      log-format      %d      [%* %H:%M:%S]   %^      %^      %^      %^      %h      %r    %^       %^      %^      %^      %^      -o      /var/www/html/ns1-web-report.html

# We want to know the following:
#       Date (1)
#       Time (2)
#       Source IP (7)
#       query (9)
#

#       -f      What log file to use
#       -o      Where to send the output report

# Sample log entry
# 1             2               3               4       5       6               7                                             89                                       10      11      12      13
# 25-Jun-2022   23:41:19.258    queries:        info:   client  @0x7f31eca19ed8 208.69.34.83#56547 (251.128-25.111.140.207.in-a
ddr.arpa):      query:  251.128-25.111.140.207.in-addr.arpa     IN      PTR     -E(0)DC (199.4.110.11)

[kpgreene]

Hello:
It seems my script when pasted turned into something nearly unreadable. I am very sorry it has happen. It seems the GitHub system bit like the pound sign used to comment a line.

I am trying to send it a different way.
I am also showing the error message.

Thank you
Kevin

[root@ns1 scripts]# ./go-dns-access.sh
 [PARSING %^] {0} @ {0/s}
Cleaning up resources...
==74189== GoAccess - Copyright (C) 2009-2020 by Gerardo Orellana
==74189== https://goaccess.io - <hello@goaccess.io>
==74189== Released under the MIT License.
==74189==
==74189== FILE: /var/log/named/queries
==74189== Parsed 10 lines producing the following errors:
==74189==
==74189== IPv4/6 is required.
==74189== IPv4/6 is required.
==74189== IPv4/6 is required.
==74189== IPv4/6 is required.
==74189== IPv4/6 is required.
==74189== IPv4/6 is required.
==74189== IPv4/6 is required.
==74189== IPv4/6 is required.
==74189== IPv4/6 is required.
==74189== IPv4/6 is required.
==74189==
==74189== Format Errors - Verify your log/date/time format

[allinurl]

It may be able to extract data from it. Could you please attach a file with some DNS entries on it? Thanks

[kpgreene]

Hi: Thank you very much for helping in this challenge I have. I have attached a ~1 hour sample of log entries (about 20 MB before creating the zip file). It would be great to have a solution. There really is no solution today for looking at the DNS (Bind9) log entries.  I spent a lot of time looking and it is really hard to find much of anything that would work unless you want to spend thousands of US dollars on a solution. The key thing is really to understand at what time and date are the people who are doing queries. What is strange to me is that a lot of my queries seem to be for an older DNS server I had and it has not been operational for many months (207.140.111.251). The question is why and where are these people doing queries of it. It seems to be from all over the place. Again, that you for taking the time to look at the challenge I have. Thank you. Kevin

…

On 7/1/2022 6:26 PM, Gerardo O. wrote: It may be able to extract data from it. Could you please attach a file with some DNS entries on it? Thanks — Reply to this email directly, view it on GitHub <#2346 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF44KV5HAYEHVJ6N77GPAELVR55EBANCNFSM52NSGSGA>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[allinurl]

It looks like the attachment was not posted. You may need to post it via github (attach files by dragging)... Thanks

[allinurl]

Anyways, this should work for bind9 query log (assuming the following format):

06-Jul-2022 15:57:29.353 queries: info: client @0x7fbff80430c0 146.112.174.76#2297 (mail.yahoo.com): query: mail.yahoo.com IN A -E(0)DC (101.170.136.1)

run goaccess as:

goaccess dns.log --log-format='%d %t.%^ %^: %^ %^ %^ %h#%^ (%v) %^ %^ %^ %U %^' --date-format=%d-%b-%Y --time-format=%T --ignore-panel=BROWSERS --ignore-panel=OS --ignore-panel=REQUESTS_STATIC --ignore-panel=NOT_FOUND --ignore-panel=REFERRING_SITES --ignore-panel=STATUS_CODES --http-protocol=no --http-method=no

[kpgreene]

Hello:

The results from your answer were very IMPRESSIVE! I used the first example that was provided.

I found that the biggest abusive group that was hitting my DNS server was out of a middle eastern country (IL). It was roughly 2,000 hits an hour for one specific IP address. I need to now look at the various IP addresses shown as the top query sources to see where they are from.

I also need to get the script to present in a web format but I think I can solve that issue (it easier to print that way).

THANK YOU AGAIN !!!

Kevin

[allinurl]

Happy to hear that worked!

For the html report, you can always pass -o report.html and --real-time-html to make it real-time.

goaccess dns.log -o report.html --real-time-html --log-format='%d %t.%^ %^: %^ %^ %^ %h#%^ (%v) %^ %^ %^ %U %^' --date-format=%d-%b-%Y --time-format=%T --ignore-panel=BROWSERS --ignore-panel=OS --ignore-panel=REQUESTS_STATIC --ignore-panel=NOT_FOUND --ignore-panel=REFERRING_SITES --ignore-panel=STATUS_CODES --http-protocol=no --http-method=no

e.g.,

Closing this, feel free to reopen it if needed.