Skip to content

fix: Further updates to Nautobot api response logging #669

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Feb 24, 2025
Merged

fix: Further updates to Nautobot api response logging #669

merged 5 commits into from
Feb 24, 2025

Conversation

stevekeay
Copy link
Contributor

Logs more information, but limits the size of any raw payloads we dump out to the logger.

Also extracts the "error" key instead of logging the whole json payload.

neutron_understack.nautobot.NautobotRequestError: Nautobot API ERROR 404 for https://nautobot.dev.undercloud.rackspace.net/testing-failures POST {'payload1': 123}: {'body': 'b\'\\n\\n\\n\\n<!DOCTYPE html>\\n<html lang="en">\\n<head>\\n    '
         '<title>Page Not Found - Nautobot</title>\\n    \\n\\n\\n\\n    <link '
         'rel="stylesheet" id="template-theme"\\n          '
         'href="/template.css">\\n    <lin...10971 more chars'}

@cardoe
Copy link
Contributor

cardoe commented Feb 18, 2025

So I'm a -1 for pretty printing in log files because we should be sending this to stuff like ELK and those log lines get parsed and split up with the leading context. As soon as you force newlines in that messes that up.

@cardoe cardoe force-pushed the nautobot-api-response-logging branch 2 times, most recently from a4c8dd9 to 7689e14 Compare February 18, 2025 15:47
@stevekeay
When logging API error responses, parse any JSON payload
c796318 
Don't log the raw JSON payload, parse it and print it in python format.
If it can't be parsed, dump the string.
@stevekeay
Explicitly log the empty body of API response with zero content-length
38c0caa
@stevekeay
Only log the error part of an API response containing an "error" key
41a92ee 
If the body consists of a json payload with a top-level key "error"
then we want to log only the contents of that key, otherwise log the
whole response.
@stevekeay
When logging API error responses, include request method and payload
6885962 
This could potentially leak sensitive information into the logs, however
we don't use the Nautobot API client to post any sensitive data and API
authentication is done via HTTP headers that are not logged.
@stevekeay
When logging raw payload of large API errors, truncate the payload
23eff58 
This is so we don't get pages of HTML in our container logs
@stevekeay stevekeay force-pushed the nautobot-api-response-logging branch from 7689e14 to 23eff58 Compare February 24, 2025 11:06
@stevekeay
Copy link
Contributor Author

So I'm a -1 for pretty printing in log files because we should be sending this to stuff like ELK and those log lines get parsed and split up with the leading context. As soon as you force newlines in that messes that up.

Fair enough, I removed the pretty-printing and rebased on main.

@stevekeay stevekeay added this pull request to the merge queue Feb 24, 2025
Merged via the queue into main with commit 645aec9 Feb 24, 2025
27 checks passed
@stevekeay stevekeay deleted the nautobot-api-response-logging branch February 24, 2025 11:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cardoe cardoe cardoe approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@stevekeay @cardoe