Snap plugin intended to process logs using regular expressions
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.


This project will no longer be maintained by Intel. Intel will not provide or guarantee development of or support for this project, including but not limited to, maintenance, bug fixes, new releases or updates. Patches to this project are no longer accepted by Intel. If you have an ongoing need to use this project, are interested in independently developing it, or would like to maintain patches for the community, please create your own fork of the project.

Snap plugin processor - logs-regexp

Snap plugin intended to process logs using regular expressions.

  1. Getting Started
  1. Documentation
  1. Community Support
  2. Contributing
  3. License
  4. Acknowledgements


Download logs-regexp plugin binary:

You can get the pre-built binaries for your OS and architecture from the plugin's GitHub Releases page. Download the plugin from the latest release and load it into snapteld (/opt/snap/plugins is the default location for Snap packages).

To build the plugin binary:


Clone repo into $GOPATH/src/github/intelsdi-x/:

$ git clone<yourGithubID>/snap-plugin-processor-logs-regexp

Build the plugin by running make in repo:

$ make

This builds the plugin in ./build

Configuration and Usage


The intention of this plugin is to parse logs according to the form defined by the user in task manifest.

The plugin processes log line using regular expressions with names. These names indicate tags which are added to metric. The regular expression can contain special names (timestamp and message) to replace metric timestamp and value.

The plugin can be configured by following parameters (all parameters are optional):

  • regexp_log - regular expression with method to parse log line, special names:
    • message - the main message in log which will be set as a metric value
    • timestamp - the log timestamp which will be set as a metric timestamp.
  • regexp_message - regular expression with method to parse the main message in log (indicated by message).
  • regexp_time - regular expression with method to parse the log timestamp (indicated by timestamp), this expression needs to contain following names: day, month, year, hour, seconds, minutes, seconds, timezone.

Notice: Special characters in regular expressions needs to be escaped.

On default plugin processes following log line (metric value which is returned by collector plugin): - - [07/Dec/2016:06:00:12 -0500] "GET /v3/users/fa2b2986c200431b8119035d4a47d420/projects HTTP/1.1" 200 446 21747 "-" "python-keystoneclient"

to Snap's metric with:

  • Data: "GET /v3/users/fa2b2986c200431b8119035d4a47d420/projects HTTP/1.1" 200 446 21747 "-" "python-keystoneclient"
  • Timestamp: 07/Dec/2016:06:00:12 -0500
  • Tags:
    • client_ip:
    • http_method:GET
    • http_url: /v3/users/fa2b2986c200431b8119035d4a47d420/projects
    • http_version: 1.1
    • http_status: 200
    • http_response_size: 446
    • http_response_time: 21747


This is an example running snap-plugin-collector-logs, processing collected regexp-logs and writing post-processed data to a file. It is assumed that you are using the latest Snap binary and plugins.

In one terminal window, open the Snap daemon (in this case with logging set to 1 and trust disabled) with appropriate configuration needed by logs collector. To do that properly, please follow the instruction on snap-plugin-collector-logs.

$ snapteld -l 1 -t 0 --config config.json

In another terminal window:

Download and load plugins:

$ wget
$ wget
$ wget
$ snaptel plugin load snap-plugin-collector-logs
$ snaptel plugin load snap-plugin-processor-logs-regexp
$ snaptel plugin load snap-plugin-publisher-file

Create a task manifest - see examplary task manifests in examples/tasks:

  "version": 1,
  "schedule": {
    "type": "simple",
    "interval": "15s"
  "workflow": {
    "collect": {
      "metrics": {
        "/intel/logs/*": {}
      "process": [
          "plugin_name": "logs-regexp",
          "config": {
            "regexp_log": "(?P<client_ip>\\S+) (\\S{1,}) (\\S{1,}) [[](?P<timestamp>\\d{2}[/]\\S+[/]\\d{4}[:]\\d{2}[:]\\d{2}[:]\\d{2} \\S\\d+)[]] (?P<message>.*)",
            "regexp_message": "(?P<http_method>[A-Z]{3,}) (?P<http_url>/\\S*) HTTP/(?P<http_version>\\d+.\\d+)\" (?P<http_status>\\d*) (?P<http_response_size>\\S*) (?P<http_response_time>\\S*)",
            "regexp_time" : "(?P<day>\\d{2})/(?P<month>[a-zA-Z]+)/(?P<year>\\d{4}):(?P<hour>\\d{2}):(?P<minutes>\\d{2}):(?P<seconds>\\d{2}) (?P<timezone>.\\d+)"
          "process": null,
          "publish": [
              "plugin_name": "file",
              "config": {
                "file": "/tmp/published_logs_with_config.log"

Create a task:

$ snaptel task create -t task-config.json

To stop task:

$ snaptel task stop <task_id>


There isn't a current roadmap for this plugin, but it is in active development. As we launch this plugin, we do not have any outstanding requirements for the next release.

If you have a feature request, please add it as an issue and/or submit a pull request.

Community Support

This repository is one of many plugins in Snap, a powerful telemetry framework. See the full project at To reach out to other users, head to the main framework


We love contributions!

There's more than one way to give back, from examples to blogs to code updates. See our recommended process in


Snap, along with this plugin, is an Open Source software released under the Apache 2.0 License.


And thank you! Your contribution, through code and participation, is incredibly important to us.