Adds bytes and entries to chunk metadata in tsdb #5414
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cyriltovena
reviewed
Feb 16, 2022
pkg/storage/tsdb/index/chunk.go
Outdated
| @@ -4,7 +4,15 @@ package index | |||
| type ChunkMeta struct { | |||
| Checksum uint32 | |||
|
|
|||
| // Time range the data covers. | |||
| // When MaxTime == math.MaxInt64 the chunk is still open and being appended to. | |||
| // Nanosecond precision | |||
There was a problem hiding this comment.
I'm intending to store nanoseconds here, but the code we have so far doesn't yet use time.Time anywhere to populate this field -- it's just an int64 so I added the comment to give more clarity.
kavirajk
reviewed
Feb 17, 2022
| // should have little effect as long as there is enough memory available | ||
| Bytes uint64 | ||
|
|
||
| Entries uint32 |
There was a problem hiding this comment.
Q: I assume this is number of log entries that are part of the particular chunk correct?. I'm trying to understand why uint32 here and how it correlates with max size of each chunk?
There was a problem hiding this comment.
Yeah, exactly. I used uint64 for bytes (rationale in the comment) but uint32 for the entries count for space reduction purposes. That make sense?
|
ref #5428 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This naively adds the number of bytes and entries for each chunk as part of it's metadata in tsdb. We may store more optimized (smaller) forms in the future, such as
KBinstead ofbytesand/or useuint32instead ofuint64to store them, but I decided to take the naive path first.