Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarks for zerolog #483

Merged
merged 3 commits into from
Jul 25, 2017
Merged

Add benchmarks for zerolog #483

merged 3 commits into from
Jul 25, 2017

Conversation

akshayjshah
Copy link
Contributor

Add some benchmarks for github.com/rs/zerolog.

These aren't exactly the same as the benchmarks in the author's PR:

  1. The author's benchmarks don't timestamp zerolog's output, which is
    why they're so startingly fast.
  2. Zerolog's sampler is too simple for reasonable production use - it
    just blindly drops a percentage of logs, rather than reservoir
    sampling. Given that, I left out sampling benchmarks.

After those two adjustments, zerolog ends up being a bit faster than zap (~50
ns) when all context has already been added, but much slower (~14000 ns) when
adding fields.

This seems like a reasonable price to pay for zap's configurability,
thread-safety, and encoding-agnosticism.

Akshay Shah added 2 commits July 24, 2017 16:17
Add some benchmarks for `github.com/rs/zerolog`.

These aren't exactly the same as the benchmarks in the author's PR:

1. The author's benchmarks don't timestamp zerolog's output, which is
   why they're so startingly fast.
2. Zerolog's sampler is too simple for reasonable production use - it
   just blindly drops a percentage of logs, rather than reservoir
   sampling. Given that, I left out sampling benchmarks.

After those two adjustments, zerolog ends up being a bit faster than zap
(~50 ns) when not adding context, but much slower (~14000 ns) when
adding fields.

This seems like a reasonable price to pay for zap's configurability,
thread-safety, and encoding-agnosticism.
@akshayjshah akshayjshah requested a review from bufdev July 24, 2017 23:21
@codecov
Copy link

codecov bot commented Jul 24, 2017

Codecov Report

Merging #483 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@          Coverage Diff           @@
##           master    #483   +/-   ##
======================================
  Coverage    96.9%   96.9%           
======================================
  Files          36      36           
  Lines        2197    2197           
======================================
  Hits         2129    2129           
  Misses         59      59           
  Partials        9       9

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a0c01aa...1c09d5b. Read the comment docs.

@bufdev
Copy link
Contributor

bufdev commented Jul 25, 2017

I feel like the README should mention that zerolog isn't thread-safe (actually zerolog's README probably should), as I still don't think this is a fair comparison - all the other logging libraries we benchmark are thread-safe as I think we'd expect, and some may want this explicit tradeoff, but it should be mentioned before we link to zerolog as an alternative.

@akshayjshah
Copy link
Contributor Author

Well...it's thread-safe as long as you use the APIs as intended. Some of the top-level objects aren't thread-safe, but I don't see that as an issue.

Rather than picking and choosing based on taste, I'm more inclined to drop seldom-used libraries. We're currently maintaining a surprisingly large amount of benchmarking code, and many of the reference libraries aren't commonly used.

@akshayjshah akshayjshah merged commit f948727 into master Jul 25, 2017
@akshayjshah akshayjshah deleted the ajs-zerolog branch July 25, 2017 14:55
@rs
Copy link

rs commented Jul 25, 2017

I just had a quick look and it seems that you are mainly using the Interface field type for most fields which is the last resort slow path for zerolog. I understand you did that for missing array support of other types. My experience with structured logging is that we almost never use these, so I’m not sure how realistic it is to use them in benchmarks. To make it fair I’m gonna add support for those types.

I did not check but the slow down you observe is most likely due to the use Interface rather than adding timestamps.

@akshayjshah
Copy link
Contributor Author

Interesting. We use structured logging in a few thousand microservices, and logging collections of user-defined types is pretty standard - applications naturally log using their own data model.

The slowdown in adding fields is likely due to using a wider variety of input types. However, the 50ns results you list on the zerolog readme for printf-style logs and static strings are largely due to the lack of timestamps; enabling timestamps makes a (relatively) large performance impact. The remainder of the delta for logging static strings is partially because of better branch prediction in the hot path (since zerolog supports fewer config options) and partially due to more aggressive inlining.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants