Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize timestamp run-length decoding #6686

Merged
merged 2 commits into from May 23, 2016
Merged

Optimize timestamp run-length decoding #6686

merged 2 commits into from May 23, 2016
+30 −22

Conversation

jwilder
Copy link
Contributor

@jwilder jwilder commented May 19, 2016 •
edited

Required for all non-trivial PRs
  • Rebased/mergable
  • Tests pass
  • CHANGELOG.md updated

This PR switched time stamps using run-length encoding to decode without allocating a large slice of values. Instead, it now returns them incrementally on demand.

Timestamp RLE decoding benchmarks

benchmark                      old ns/op     new ns/op     delta
BenchmarkTimeDecoder_RLE-8     6319          2339          -62.98%

benchmark                      old allocs     new allocs     delta
BenchmarkTimeDecoder_RLE-8     0              0              +0.00%

benchmark                      old bytes     new bytes     delta
BenchmarkTimeDecoder_RLE-8     0             0             +0.00%

Block decoding benchmarks using RLE

benchmark                                       old ns/op     new ns/op     delta
BenchmarkDecodeBlock_Float_Empty-8              42530         35008         -17.69%
BenchmarkDecodeBlock_Float_EqualSize-8          39902         32206         -19.29%
BenchmarkDecodeBlock_Float_TypeSpecific-8       29570         22632         -23.46%
BenchmarkDecodeBlock_Integer_Empty-8            32222         24617         -23.60%
BenchmarkDecodeBlock_Integer_EqualSize-8        29290         21661         -26.05%
BenchmarkDecodeBlock_Integer_TypeSpecific-8     18852         11423         -39.41%
BenchmarkDecodeBlock_Boolean_Empty-8            27943         20535         -26.51%
BenchmarkDecodeBlock_Boolean_EqualSize-8        25809         18142         -29.71%
BenchmarkDecodeBlock_Boolean_TypeSpecific-8     15100         7427          -50.81%
BenchmarkDecodeBlock_String_Empty-8             95612         87524         -8.46%
BenchmarkDecodeBlock_String_EqualSize-8         92865         84970         -8.50%
BenchmarkDecodeBlock_String_TypeSpecific-8      78228         69231         -11.50%

benchmark                                       old allocs     new allocs     delta
BenchmarkDecodeBlock_Float_Empty-8              24             13             -45.83%
BenchmarkDecodeBlock_Float_EqualSize-8          23             12             -47.83%
BenchmarkDecodeBlock_Float_TypeSpecific-8       12             1              -91.67%
BenchmarkDecodeBlock_Integer_Empty-8            25             14             -44.00%
BenchmarkDecodeBlock_Integer_EqualSize-8        24             13             -45.83%
BenchmarkDecodeBlock_Integer_TypeSpecific-8     13             2              -84.62%
BenchmarkDecodeBlock_Boolean_Empty-8            24             13             -45.83%
BenchmarkDecodeBlock_Boolean_EqualSize-8        23             12             -47.83%
BenchmarkDecodeBlock_Boolean_TypeSpecific-8     12             1              -91.67%
BenchmarkDecodeBlock_String_Empty-8             1025           1014           -1.07%
BenchmarkDecodeBlock_String_EqualSize-8         1024           1013           -1.07%
BenchmarkDecodeBlock_String_TypeSpecific-8      1013           1002           -1.09%

benchmark                                       old bytes     new bytes     delta
BenchmarkDecodeBlock_Float_Empty-8              67562         51186         -24.24%
BenchmarkDecodeBlock_Float_EqualSize-8          51178         34801         -32.00%
BenchmarkDecodeBlock_Float_TypeSpecific-8       18424         2048          -88.88%
BenchmarkDecodeBlock_Integer_Empty-8            69610         53234         -23.53%
BenchmarkDecodeBlock_Integer_EqualSize-8        53226         36849         -30.77%
BenchmarkDecodeBlock_Integer_TypeSpecific-8     20472         4096          -79.99%
BenchmarkDecodeBlock_Boolean_Empty-8            67562         51186         -24.24%
BenchmarkDecodeBlock_Boolean_EqualSize-8        51177         34801         -32.00%
BenchmarkDecodeBlock_Boolean_TypeSpecific-8     18424         2048          -88.88%
BenchmarkDecodeBlock_String_Empty-8             109644        93267         -14.94%
BenchmarkDecodeBlock_String_EqualSize-8         93259         76883         -17.56%
BenchmarkDecodeBlock_String_TypeSpecific-8      44121         27745         -37.12%

@benbjohnson

@jwilder jwilder added this to the 1.0.0 milestone May 19, 2016
@jwilder jwilder changed the title Optimized timestamp run-length decoding Optimize timestamp run-length decoding May 19, 2016
benbjohnson
benbjohnson reviewed May 20, 2016
View reviewed changes
tsdb/engine/tsm1/timestamp.go
err error
v int64
i, n int
ts []uint64
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is ts used anymore?

@benbjohnson
Copy link
Contributor

One comment otherwise 👍

@jwilder
Copy link
Contributor Author

jwilder commented May 20, 2016

Still used by simple8b decoder. I'll remove that next.

@e-dard
Copy link
Contributor

e-dard commented May 20, 2016

LGTM 👍

@jwilder
Optimized timestamp run-length decoding
f48a106 
Removes the up-front allocation of decoded values and return them
as needed.
@jwilder
Update changelog
eafe6ca
@jwilder jwilder merged commit 0752ca8 into master May 23, 2016
@jwilder jwilder deleted the jw-timestamp branch May 23, 2016 21:24
@timhallinflux timhallinflux modified the milestones: 1.0.0, 1.0.0 beta Dec 19, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
area/performance area/tsm
Projects
None yet
Milestone
1.0.0
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@jwilder @benbjohnson @e-dard @timhallinflux