Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

prometheus-nginxlog-exporter made wrong regex for given format #378

Open
JoobyPM opened this issue Feb 14, 2024 · 0 comments
Open

prometheus-nginxlog-exporter made wrong regex for given format #378

JoobyPM opened this issue Feb 14, 2024 · 0 comments
Labels

Comments

@JoobyPM
Copy link

JoobyPM commented Feb 14, 2024

Describe the bug
prometheus-nginxlog-exporter made wrong regex for given format.
Given nginx log format:

log_format  main  '$http_authorization bbs=$body_bytes_sent bs=$bytes_sent rl=$request_length rt=$request_time urt=$upstream_response_time $remote_addr - $http_x_forwarded_for $remote_user [$time_local] "$request" $status "$http_referer" "$http_user_agent"';

Here is explained regex101 - https://regex101.com/r/pjD94u/1

Does not match given format

^(?P<http_authorization>[^ ]*) bbs=(?P<body_bytes_sent>[^ ]*) bs=(?P<bytes_sent>[^ ]*) rl=(?P<request_length>[^ ]*) rt=(?P<request_time>[^ ]*) urt=(?P<upstream_response_time>[^ ]*) (?P<remote_addr>[^ ]*) - (?P<http_x_forwarded_for>[^ ]*) (?P<remote_user>[^ ]*) \[(?P<time_local>[^]]*)\] "(?P<request>[^"]*)" (?P<status>[^ ]*) "(?P<http_referer>[^"]*)" "(?P<http_user_agent>[^"]*)

$http_authorization - might have values Bearer\s\S+|Basic\s\S+|-

To Reproduce

  1. Add to configuration of nginx log format http_authorization
  2. Send some request with JWT (Bearer ...)
  3. Check error log of prometheus-nginxlog-exporter

Expected behavior
For a given format, the correct regex will be:

^(?<http_authorization>\S+\s\S+|-) bbs=(?<body_bytes_sent>\d*|-) bs=(?<bytes_sent>\d*|-) rl=(?<request_length>\d*|-) rt=(?<request_time>[\d.]*|-) urt=(?<upstream_response_time>[\d.]*|-) (?<remote_addr>-|(?:\d{1,3}(?:\.\d{1,3}){3})|(?:[0-9a-fA-F:]{2,39})) - (?<http_x_forwarded_for>-|(?:\d{1,3}(?:\.\d{1,3}){3})|(?:[0-9a-fA-F:]{2,39})) (?<remote_user>[\S-]+) \[(?<time_local>[^\]]+)\] "(?<method>\S+) (?<url>.*?) (?<protocol>HTTP\/\d\.\d)" (?<status>\d{3}) "(?<http_referer>[^"]*|-)" "(?<http_user_agent>[^"]*)"$

For $http_authorization - (?<http_authorization>\S+\s\S+|-) instead of (?P<http_authorization>[^ ]*)

Log file:

Feb 14 06:26:03 server-frontend nginx_exporter[9955]: 2024-02-14T06:26:03.036-0300#011error#011prometheus-nginxlog-exporter/main.go:302#011error while parsing line 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c bbs=105 bs=318 rl=1139 rt=0.183 urt=0.184 111.111.111.111 - - - [14/Feb/2024:06:26:02 -0300] "GET /app-api/ping HTTP/1.1" 200 "-" "Mozilla/5.0 (Web0S; Linux/SmartTV) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.34 Safari/537.36 WebAppManager"'
: text log parsing err: access log line 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c bbs=105 bs=318 rl=1139 rt=0.183 urt=0.184 111.111.111.111 - - - [14/Feb/2024:06:26:02 -0300] "GET /app-api/ping HTTP/1.1" 200 "-" "Mozilla/5.0 (Web0S; Linux/SmartTV) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.34 Safari/537.36 WebAppManager"' does not match given format 
 ^(?P<http_authorization>[^ ]*) bbs=(?P<body_bytes_sent>[^ ]*) bs=(?P<bytes_sent>[^ ]*) rl=(?P<request_length>[^ ]*) rt=(?P<request_time>[^ ]*) urt=(?P<upstream_response_time>[^ ]*) (?P<remote_addr>[^ ]*) - (?P<http_x_forwarded_for>[^ ]*) (?P<remote_user>[^ ]*) \[(?P<time_local>[^]]*)\] "(?P<request>[^"]*)" (?P<status>[^ ]*) "(?P<http_referer>[^"]*)" "(?P<http_user_agent>[^"]*)"

Environment:

  • Exporter version: v1.11.0
  • OS (e.g. from /etc/os-release): Ubuntu 22.04
  • Deployment method (e.g. Docker image, deb/rpm package, self-compiled, ...): deb/rpm package
@JoobyPM JoobyPM added the bug label Feb 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant