Utility for parsing Bro log files into CSV or JSON format
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
examples
parsebrologs Correctly escaped double quotes in field value Jun 30, 2018
LICENSE.txt
README.md
setup.cfg
setup.py

README.md

ParseBroLogs

A lightweight utility for programmatically reading and manipulating Bro IDS log files and outputting into JSON or CSV format. This library works on both Python 2 and Python 3.

Examples

The following example first loads records from the Bro connection log named conn.log. The data is the written out to a file named out.json. The name of the log file to read must be provided when creating the ParseBroLog class. You can use the safe_headers=True option in the to_json method to replace all instances of a dot with an underscore.

from parsebrologs import ParseBroLogs

log_data = ParseBroLogs("conn.log")
with open('out.json',"w") as outfile:
    outfile.write(log_data.to_json())

This is another example that instead uses the to_csv method to write the data out to a file named out.csv. This example shows filtering on specific fields within the log file. Field names should be provided as list elements.

from parsebrologs import ParseBroLogs

log_data = ParseBroLogs("conn.log", fields=["ts", "id.orig_h", "id.resp_h"])
with open('out.csv',"w") as outfile:
    outfile.write(log_data.to_csv())

If you are planning to open the csv using Microsoft Excel or OpenOffice, you might want to use the to_escaped_csv() method. This adds quotes around the data escaping any commas or other special characters that cause problems with csv viewers.

from parsebrologs import ParseBroLogs

log_data = ParseBroLogs("conn.log"])
with open('out.csv','w') as outfile:
    outfile.write(log_data.to_escaped_csv())

If you are planning on using pandas to manipulate the data, you can use the to_raw_data method directly with Pandas constructor. Because the to_json() method returns the json data as a string, you should use the json library to convert out of string format.

from parsebrologs import ParseBroLogs
import pandas as pd
import json

log_data = ParseBroLogs("conn.log", fields=["ts", "id.orig_h", "id.resp_h"])
df = pd.DataFrame(json.loads(log_data.to_json()))
df

Special Thanks

  • @geekscrapy: For bug fixes and the safe header feature addition