New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce logging by removing content duplication #885

Merged
merged 2 commits into from Jun 11, 2018

Conversation

3 participants
@rcillo
Member

rcillo commented Jun 5, 2018

https://techjira.zalando.net/browse/ARUHA-1698

On each request to publish an event to Nakadi, 2 lines of logs are
generated: 1 for ACCESS_LOG and 1 for SLO log.

SLO log is used by scalyr to report SLOs. It contains basically the same
information but with the addition of the request size.

This change adds the body size to ACCESS_LOG and removes SLO for
publishing requests. This will result in an overall reduction of 30% of log lines,
which is the amount of logging we were requested to reduce in order to
avoid overspending on logs.

Reduce logging by removing content duplication
On each requests to publish an event to Nakadi, 2 lines of logs were
generated: 1 for ACCESS_LOG and 1 for SLO log.

SLO log is used by scalyr to report SLO. It contains basically the same
information but with the addition of the request size in a more
structured way.

This change adds the body size to ACCESS_LOG and removes SLO for
publishing requests. This will result in an overall reduction of 30%,
which is the amount of logging we were requested to reduce in order to
avoid overspending on logs.
@codecov-io

This comment has been minimized.

codecov-io commented Jun 5, 2018

Codecov Report

Merging #885 into master will increase coverage by <.01%.
The diff coverage is 0%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #885      +/-   ##
============================================
+ Coverage     53.62%   53.63%   +<.01%     
- Complexity     1705     1706       +1     
============================================
  Files           314      314              
  Lines          9501     9500       -1     
  Branches        857      858       +1     
============================================
  Hits           5095     5095              
  Misses         4098     4098              
+ Partials        308      307       -1
Impacted Files Coverage Δ Complexity Δ
...o/nakadi/controller/EventPublishingController.java 85.71% <ø> (-0.4%) 16 <0> (ø)
...java/org/zalando/nakadi/filters/LoggingFilter.java 16.12% <0%> (-0.54%) 2 <0> (ø)
...n/java/org/zalando/nakadi/service/EventStream.java 75% <0%> (+1.61%) 30% <0%> (+1%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9d116a2...6a77d75. Read the comment docs.

@lmontrieux

This comment has been minimized.

Member

lmontrieux commented Jun 5, 2018

👍

@rcillo

This comment has been minimized.

Member

rcillo commented Jun 6, 2018

deploy validation please

@rcillo

This comment has been minimized.

Member

rcillo commented Jun 7, 2018

deploy validation please

@rcillo

This comment has been minimized.

Member

rcillo commented Jun 11, 2018

👍

1 similar comment
@lmontrieux

This comment has been minimized.

Member

lmontrieux commented Jun 11, 2018

👍

@rcillo rcillo merged commit f55117d into master Jun 11, 2018

7 of 8 checks passed

codecov/patch 0% of diff hit (target 53.62%)
Details
Codacy/PR Quality Review Up to standards. A positive pull request.
Details
aruha/jenkins Build finished.
Details
codecov/project 53.63% (+<.01%) compared to 9d116a2
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
zappr Approvals: @rcillo, @lmontrieux.
zappr/pr/specification PR has passed specification checks

@rcillo rcillo deleted the reduce-logging branch Jun 11, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment