Skip to content

add response body logging to non 2xx responses #142

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Jun 4, 2024
Merged

add response body logging to non 2xx responses #142

merged 5 commits into from
Jun 4, 2024

Conversation

jsvd
Copy link
Member

@jsvd jsvd commented May 9, 2024 •
edited
Loading

Log the response body and the event (in JSON format) when the endpoint returns a non 2xx response.

Before, the logging entry included :event => event, but the event wasn't being displayed at all, it was just an stringified object reference. Given the potential size of the event or response size, both of these are only logged in debug mode.
This PR makes logging more consistent with the retryable error logging.

jsvd
jsvd commented May 9, 2024
View reviewed changes
andsel
andsel requested changes May 9, 2024
View reviewed changes
Copy link

@andsel andsel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also add a section in the doc describing this behavior

jsvd
jsvd commented May 10, 2024
View reviewed changes
@jsvd jsvd changed the title add response body and event logging to non 2xx responses add response body logging to non 2xx responses May 10, 2024
jsvd
jsvd commented Jun 3, 2024
View reviewed changes
@jsvd jsvd requested a review from andsel June 3, 2024 12:53
@jsvd
Copy link
Member Author

jsvd commented Jun 3, 2024

@andsel I just simplified the PR for now to add the logging of the body. I also removed the event logging since it was broken already. it can be added in a separate PR.

@andsel
Copy link

andsel commented Jun 3, 2024 •
edited
Loading

Tested with simple failing pipeline:

input {
  generator {
    message => '{"greetings": "Hello world!"}'
    count => 1
    codec => json
  }
}

output {
  http {
    url => "https://google.it/sink"
    http_method => "post"
  }
}

Log before

[2024-06-03T15:27:23,040][ERROR][logstash.outputs.http    ][main][http_out] Encountered non-2xx HTTP code 404 
{:response_code=>404, :url=>"https://google.it/sink", :event=>#<LogStash::Event:0x6992dbab>}

Log after

2024-06-03T15:35:30,184][ERROR][logstash.outputs.http    ][main][http_out] Encountered non-2xx HTTP code 
{:code=>404, :body=>"<!DOCTYPE html>\n<html lang=en>\n  <meta charset=utf-8>\n  <meta name=viewport content=\"initial-scale=1, minimum-scale=1, width=device-width\">\n  
<title>Error 404 (Not Found)!!1</title>\n  
<style>\n    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}\n  
</style>\n  
<a href=//www.google.com/><span id=logo aria-label=Google></span></a>\n  
<p><b>404.</b> <ins>That’s an error.</ins>\n 
 <p>The requested URL <code>/sink</code> was not found on this server.  <ins>That’s all we know.</ins>\n", :url=>"https://google.it/sink"
}

andsel
andsel approved these changes Jun 3, 2024
View reviewed changes
Copy link

@andsel andsel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after:

  • CI is green
  • update version also in gemspec

@jsvd jsvd merged commit e746509 into logstash-plugins:main Jun 4, 2024
2 checks passed
@jsvd jsvd deleted the error_400_logging branch June 4, 2024 09:48
@jsvd jsvd mentioned this pull request Jun 4, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andsel andsel andsel approved these changes

Assignees

@jsvd jsvd

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jsvd @andsel @roaksoax