Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support haproxy logs without captured headers #9463

Closed
jsoriano opened this issue Dec 10, 2018 · 3 comments
Closed

Support haproxy logs without captured headers #9463

jsoriano opened this issue Dec 10, 2018 · 3 comments

Comments

@jsoriano
Copy link
Member

@jsoriano jsoriano commented Dec 10, 2018

haproxy logs don't contain captured headers by default, but current grok patterns expect them, make them optional and add test cases for them.

Some examples:

Without captured headers:

Dec 10 12:01:46 voyager haproxy[19312]: 127.0.0.1:35982 [10/Dec/2018:12:01:46.395] http-webservices http-webservices/<NOSRV> 0/-1/-1/-1/0 503 213 - - SC-- 1/1/0/0/0 0/0 "GET / HTTP/1.1"

With request captured headers:

Dec 10 15:46:49 voyager haproxy[29785]: 127.0.0.1:43738 [10/Dec/2018:15:46:49.497] http-webservices http-webservices/<NOSRV> 0/-1/-1/-1/0 503 213 - - SC-- 1/1/0/0/0 0/0 {localhost:8888||} "GET /foo HTTP/1.1"

With request and response captured headers:

Dec 10 15:48:56 voyager haproxy[7873]: 127.0.0.1:44542 [10/Dec/2018:15:48:56.017] http-webservices http-webservices/<NOSRV> 0/-1/-1/-1/0 503 213 - - SC-- 1/1/0/0/0 0/0 {localhost:8888||} {|} "GET /foo HTTP/1.1"

Headers capture is configured in haproxy config with options like:

  capture request header Host len 15
  capture request header X-Forwarded-For len 15
  capture request header Referer len 15
  capture response header Content-length len 9
  capture response header Location len 15

Notice that when only one type of headers is being captured it is not possible to know on default log formats if they are request or response headers. Maybe we should ignore them by default.

@elasticmachine

This comment has been minimized.

Copy link
Collaborator

@elasticmachine elasticmachine commented Dec 10, 2018

Pinging @elastic/infrastructure

@sentient

This comment has been minimized.

Copy link
Contributor

@sentient sentient commented Jan 15, 2019

Would it not just be simpler to document in the haproxy module the log-format you are expecting
And give us the log_format string that we can put in haproxy.cfg ?

@jsoriano

This comment has been minimized.

Copy link
Member Author

@jsoriano jsoriano commented Jan 16, 2019

@sentient actually it is also in our plans to document supported formats (see #8311)

The idea is to support the default haproxy formats (no log options, option httplog and option tcplog).

The issue addressed here is that the log format used by option httplog includes an optional part that depends on other settings.

If you are having this issue, an option to workaround it is to add captured headers, even if they don't exist, something like:

  capture request header Host len 15
  capture response header Content-length len 9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.