Logparser captures cannot contain non-word characters

[kingsleyadam]

Bug report

I’m trying to parse through the /var/log/auth.log file to extract different auth logging. Elastic even has a Blog post about this: https://www.elastic.co/blog/grokking-the-linux-authorization-logs1

Everything seems to work in https://grokdebug.herokuapp.com/1 but when I input everything into telegraf I receive no results. Anybody have any ideas?

Confirmed telegraf as access to the adm group which has access to the logs.

Relevant telegraf.conf:

[[inputs.logparser]]
    files = ["/var/log/auth.log"]
    from_beginning = false
    watch_method = "inotify"
    [inputs.logparser.grok]
      patterns = ['''%{SYSLOGTIMESTAMP:system.auth.timestamp} %{SYSLOGHOST:system.auth.hostname} sshd(?:\[%{POSINT:system.auth.pid}\])?: %{DATA:system.auth.ssh.event} user %{DATA:system.auth.user} from %{IPORHOST:system.auth.ip}''']
      measurement = "auth_log"

System info:

Telegraf v1.5.3 (git: release-1.5 1e51969)
Ubuntu Server 16.0.4
User: telegraf
Related Groups (for log access) - Logs are own by group adm: adm

Examples Logs being parsed:

Apr 10 05:11:57 localhost sshd[22041]: Invalid user frank from 172.31.14.87
Apr 10 05:11:57 localhost sshd[22041]: input_userauth_request: invalid user frank [preauth]
Apr 10 05:17:38 localhost sshd[33668]: Invalid user frank from 172.31.14.87
Apr 10 05:17:38 localhost sshd[33668]: input_userauth_request: invalid user frank [preauth]
Apr 10 05:18:36 localhost sshd[35700]: Invalid user frank from 172.31.14.87
Apr 10 05:18:36 localhost sshd[35700]: input_userauth_request: invalid user frank [preauth]

Steps to reproduce:

1. Create a log file with the above example log (make sure telegraf has access)
2. Add log parser settings as telegraf input.
3. Set telegraf output to file (e.g. stdout)
4. Start telegraf, notice nothing in stdout upon start. Nothing we new entry it added to the log.

Expected behavior:

When a new log entry is added to /var/log/auth.log I'd expect metrics to appear in stdout with measurement name "auth_log".

Actual behavior:

Nothing appears in stdout even when new log entries are created.

[danielnelson]

The pattern above does not work because the version of grok we are using does not support non-word \w characters. Support for this was added recently in vjeantet/grok#23

[danielnelson]

This issue needs resolved before we can add support for . in grok captures: vjeantet/grok#24

[sjwang90]

Closed in #8256