Passive DNS collection using Bro
Go JavaScript Other
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
assets initial web UI Aug 2, 2017
test_data refactor readers, add json implementation Sep 1, 2017
.gitignore initial go code for parsing bro logs and aggregating dns records Apr 1, 2016
.travis.yml problem: testing old go versions Feb 18, 2017
Makefile add rpm target to Makefile May 19, 2016
README.md fix typo Mar 31, 2017
aggregate.go problem: skipped records not counted Sep 1, 2017
aggregate_test.go use floats for timestamps instead of strings May 16, 2016
assets.go initial web UI Aug 2, 2017
dns-ans-query.bro initial import Feb 25, 2014
local_test.txt
main.go Un-hard code the max days value Sep 1, 2017
read.go problem: invalid records abort processing Sep 1, 2017
read_ascii.go problem: invalid records abort processing Sep 1, 2017
read_json.go problem: invalid records abort processing Sep 1, 2017
read_test.go refactor readers, add json implementation Sep 1, 2017
store.go problem: store references sql.Tx Feb 18, 2017
store_pg.go Revert "change postgresql importing to use a staging table" May 17, 2017
store_sql_common.go Revert "change postgresql importing to use a staging table" May 17, 2017
store_sqlite.go problem: store references sql.Tx Feb 18, 2017
store_test.go problem: bench doesn't complain if newstore fails May 17, 2017
version.go bump version Aug 2, 2017
web.go initial web UI Aug 2, 2017

README.md

Passive DNS for Bro

Build Status

This is an extremely simple implementation of a passive DNS collection system that utilizes Bro for DNS log collection.

Passive DNS collection can be used for various security or troubleshooting purposes. Many queries to raw DNS logs can be done faster by using the aggregated data in the passive DNS database, which is more compact.

This tool uses the Bro DNS logs to build a database of unique query+type+answer tuples.

It produces a table like this:

pdns=# select * from dns where answer='74.125.225.18' order by last desc limit 4;
      query       | type |    answer     | count | ttl |     first     |    last
------------------+------+---------------+-------+-----+---------------+------------
 www.google.com   | A    | 74.125.225.18 |  7517 | 198 | 2014-09-03 .. | 2014-10-30 ..
 t0.gstatic.com   | A    | 74.125.225.18 |   266 | 300 | 2014-09-03 .. | 2014-10-30 ..
 googlegroups.com | A    | 74.125.225.18 |   266 | 300 | 2014-09-03 .. | 2014-10-30 ..
 t3.gstatic.com   | A    | 74.125.225.18 |   291 | 300 | 2014-09-03 .. | 2014-10-30 ..

This is helpful because the PTR record itself for 74.125.225.18 is ord08s12-in-f18.1e100.net.

Examples of questions this database can answer faster than the raw logs:

  • Did anything ever resolve example.com, and if so, when was the first time?
  • What IPs has example.com resolved to?
  • What other names resolve to this IP?

Requirements

  • go compiler ( to build )
  • postgresql ( optional )

Build

$ go build

Index logs

export PDNS_STORE_TYPE="postgresql"
export PDNS_STORE_URI="postgres://pdns:foo@localhost/pdns?sslmode=disable"

# or 
export PDNS_STORE_TYPE="sqlite"
export PDNS_STORE_URI="/path/to/passivedns.sqlite"

find /usr/local/bro/logs -name dns* | sort -n | xargs -n 50 bro-pdns index

Query Database

# suffix search:
$ bro-pdns like tuples google.com
$ bro-pdns like individual google.com

# exact match
$ bro-pdns find tuples google.com
$ bro-pdns find individual google.com

Start HTTP server

$ bro-pdns web

Query HTTP API

$ curl localhost:8080/dns/like/tuples/google.com
$ curl localhost:8080/dns/like/individual/google.com
$ curl localhost:8080/dns/find/tuples/google.com
$ curl localhost:8080/dns/find/individual/google.com