Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Zstd decoder #948

Merged
merged 16 commits into from
Sep 15, 2021
Merged

Zstd decoder #948

merged 16 commits into from
Sep 15, 2021

Conversation

joe-elliott
Copy link
Member

@joe-elliott joe-elliott commented Sep 13, 2021 •
edited

What this PR does:

  • Use the zstd decoder instead of the streaming io.Reader for reads. Seeing a 10-20% increase in throughput when iterating a block and 10-20% decrease in compactor CPU with this change.
  • Update klauspost/compress dependency to latest

I also attempted the same change for the encoder/writing, but saw a performance regression so I left it out.

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@joe-elliott
blerg
481b1b0 
Signed-off-by: Joe Elliott <number101010@gmail.com>

sadness

Signed-off-by: Joe Elliott <number101010@gmail.com>

removed encoding

Signed-off-by: Joe Elliott <number101010@gmail.com>

duh

Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott
updated compression library
9bf8582 
Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott
different horribleness
ec6f5c2 
Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott
klauspost
9ef685e 
Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott
cgo zstd
04bae4a
@joe-elliott
Revert "cgo zstd"
ab35148 
This reverts commit 04bae4a.
@joe-elliott
http conn mucking
4b1e3cd 
Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott
encoder
b081970 
Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott
Revert "http conn mucking"
f312600 
This reverts commit 4b1e3cd.
@joe-elliott
Revert "encoder"
c7e9a6b 
This reverts commit b081970.
@joe-elliott
comments
1d83608 
Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott
changelog
b36f23a 
Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott
Merge branch 'main' into zstd-decoder
509501c
@joe-elliott
Merge branch 'main' into zstd-decoder
8dee031
@joe-elliott
remove test code
e95ee03 
Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott
todo
6293c2c 
Signed-off-by: Joe Elliott <number101010@gmail.com>
mdisibio
mdisibio approved these changes Sep 15, 2021
View reviewed changes
Copy link
Contributor

@mdisibio mdisibio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lg (enough) tm !

@joe-elliott joe-elliott merged commit e976a08 into grafana:main Sep 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mdisibio mdisibio mdisibio approved these changes

@annanay25 annanay25 Awaiting requested review from annanay25 annanay25 is a code owner

@dgzlopes dgzlopes Awaiting requested review from dgzlopes

@kvrhdn kvrhdn Awaiting requested review from kvrhdn kvrhdn is a code owner

@mapno mapno Awaiting requested review from mapno mapno is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@joe-elliott @mdisibio