Skip to content
Permalink
Browse files

Added support to add JSON-logs for all filters.

  • Loading branch information...
J-Gras committed Nov 16, 2017
1 parent 3a1a7e3 commit 4dcb71ffbe0b7779b4431ef59578b022013e0c21
Showing with 46 additions and 25 deletions.
  1. +20 −9 README.md
  2. +26 −16 scripts/add-json.bro
@@ -1,25 +1,36 @@
# Add-JSON

This package provides additional JSON-logging for Bro. By default a JSON log is enabled
for every logging stream (original filename suffixed by `-json`). For further configuration,
the following options are available:
This package provides additional JSON-logging for Bro. By default a JSON log is enabled for every
logging stream (original filename suffixed by `-json`). For further configuration, the following
options are available:

Option | Default Value | Description
-----------------------------|---------------------|-----------------------------------------------
-----------------------------|---------------------|---------------------------------------------------
`enable_all_json: bool` | `T` | Enables JSON-logfiles for all active streams
`enable_all_filters_json` | `F` | Enables JSON-logfiles for all filters of a stream
`exclude_json: set[Log::ID]` | `{ }` | Streams **not** to generate JSON-logfiles for
`include_json: set[Log::ID]` | `{ }` | Streams to generate JSON-logfiles for
`path_json: string` | default path | Path to the additional JSON-logfiles
`interv_json: interval` | default interval | Rotation interval for JSON-logfiles
`timestamps_json: string` | `"JSON::TS_MILLIS"` | Format of timestamps for JSON-logfiles.
`scope_sep_json: string` | default separator | Separator for log field scopes.

If, for example, your postprocessing of the files cannot handle dots in field names, you can
add the following to you `local.bro` to replace them with underscores:
If, for example, the postprocessing of JSON-logs cannot handle dots in field names, the following can
be added to `local.bro`, to replace dots with underscores:

redef Log::scope_sep_json = "_";
For more details on the underlying filter options see:
https://www.bro.org/sphinx/scripts/base/frameworks/logging/main.bro.html#type-Log::Filter
For more details on the underlying filter options see [Bro's documentation
](https://www.bro.org/sphinx/scripts/base/frameworks/logging/main.bro.html#type-Log::Filter)
of the Logging Framework.
**Note:** The script has been tested with Bro version 2.5.
## Testing
Tests using Bro's `btest` are available in a separate branch `tests`. The tests can be run manually
or automated during installation with bro-pkg (`bro-pkg install add-json --version tests`).
## Custom Logs
The add-json package sets up additional filters for the configured logs during initialization. As
the corresponding `bro_init` event handler is executed with a priority of -3, everything (streams
and filters) setup with a _higher_ priority than -3 will be considered by the script.
@@ -5,6 +5,8 @@ module Log;
export {
## Enables JSON-logfiles for all active streams
const enable_all_json = T &redef;
## Enables JSON-logfiles for all filters of a stream
const enable_all_filters_json = F &redef;
## Streams not to generate JSON-logfiles for
const exclude_json: set[Log::ID] = { } &redef;
## Streams to generate JSON-logfiles for
@@ -37,22 +39,30 @@ event bro_init() &priority=-3
["use_json"] = "T",
["json_timestamps"] = timestamps_json);

# Add filter for JSON output
for ( id in Log::active_streams )
local filters_copy = copy(Log::filters);
for ( [id, filter_name] in filters_copy )
{
if ( (enable_all_json || (id in include_json)) && (id !in exclude_json) )
{
local filter = copy(Log::get_filter(id, "default"));
filter$name = "default_json";
filter$writer = Log::WRITER_ASCII;
if ( filter?$path )
filter$path = string_cat(path_json, filter$path, "-json");
if ( filter?$path_func )
filter$path_func = json_path_func;
filter$config = config_json;
filter$interv = interv_json;
filter$scope_sep = scope_sep_json;
Log::add_filter(id, filter);
}
if ( !(enable_all_json || (id in include_json)) || (id in exclude_json) )
next; # Ignore unwanted logstreams

if ( !enable_all_filters_json && filter_name != "default" )
next; # Ignore unwanted filters

local filter = filters_copy[id, filter_name];
if ( filter$writer == Log::WRITER_ASCII && "use_json" in filter$config &&
filter$config["use_json"] == "T")
next; # Ignore existing JSON filters

# Add new filter for JSON output (previously copied)
filter$name = string_cat(filter$name, "_json");
filter$writer = Log::WRITER_ASCII;
if ( filter?$path )
filter$path = string_cat(path_json, filter$path, "-json");
if ( filter?$path_func )
filter$path_func = json_path_func;
filter$config = config_json;
filter$interv = interv_json;
filter$scope_sep = scope_sep_json;
Log::add_filter(id, filter);
}
}

0 comments on commit 4dcb71f

Please sign in to comment.
You can’t perform that action at this time.