Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix access log parsing for custom log format #346

Merged
merged 6 commits into from
Jun 28, 2023

Conversation

nkashiv
Copy link
Contributor

@nkashiv nkashiv commented Jun 12, 2023

Proposed changes

This PR aims to fix the bug where if a user has an access log format with a variable that is not present in the logVarMap https://github.com/nkashiv/agent/blob/main/src/core/metrics/sources/nginx_access_log.go#L33 then the access log parsing fails and no access log metric is collected.

For example, the following log format will cause an error because it contains $upstream_addr which is not present in the map. A user can also have a custom variable defined which will break the parsing.

log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for" '
                      '"$upstream_addr" "$upstream_response_time" "$upstream_status" '
                      '$http_x_amplify_id';

Added a unit test in nginx_access_log_test.go to demonstrate this fix.

Checklist

Before creating a PR, run through this checklist and mark each as complete.

  • I have read the CONTRIBUTING document
  • I have run make install-tools and have attached any dependency changes to this pull request
  • If applicable, I have added tests that prove my fix is effective or that my feature works
  • If applicable, I have checked that any relevant tests pass after adding my changes
  • If applicable, I have updated any relevant documentation (README.md)
  • If applicable, I have tested my cross-platform changes on Ubuntu 22, Redhat 8, SUSE 15 and FreeBSD 13

@netlify
Copy link

netlify bot commented Jun 12, 2023

Deploy Preview for agent-public-docs canceled.

Name Link
🔨 Latest commit ec6aef3
🔍 Latest deploy log https://app.netlify.com/sites/agent-public-docs/deploys/6489fb1b6257b400086df23a

@nkashiv nkashiv added the bug Something isn't working label Jun 12, 2023
@nkashiv nkashiv changed the title Access log parsing Fix access log parsing for custom log format Jun 12, 2023
@nkashiv nkashiv marked this pull request as draft June 12, 2023 22:43
@github-actions github-actions bot added chore Pull requests for routine tasks dependencies labels Jun 12, 2023
@nkashiv nkashiv marked this pull request as ready for review June 13, 2023 00:00
Copy link
Contributor

@achawla2012 achawla2012 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

src/core/metrics/sources/nginx_access_log.go Outdated Show resolved Hide resolved
@dhurley dhurley removed dependencies chore Pull requests for routine tasks labels Jun 28, 2023
@dhurley dhurley merged commit 4716d2a into nginx:main Jun 28, 2023
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants