Reorder series key tagset #12391
Merged
Reorder series key tagset #12391
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
e-dard
requested review from
nathanielc and
benbjohnson
and removed request for
stuartcarnie
benbjohnson
approved these changes
Mar 6, 2019
zeebo
reviewed
Mar 6, 2019
nathanielc
approved these changes
Mar 6, 2019
zeebo
approved these changes
Mar 6, 2019
e-dard
force-pushed
the
er-tags
branch
2 times, most recently
from
3d3022b
to
183741d
Compare
nathanielc
approved these changes
Mar 6, 2019
We will want to validate that all tag key/value data is valid unicode. This commit changes the validation helper to only validate provided tags, since measurements are currently very likely to contain invalid utf-8 characters. There are two exceptions to the tag validation: the validation of the special tag keys for measurements and field keys.
The storage engine will now drop any points that contain invalid tag data. Special tag keys for the measurement and field key will be excepted from this validation.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR makes a breaking change to the 2.x line.
Currently we use the special tag keys
_mand_fto index the measurement name and field key.This results in keys ordered like this:
With this change we will now use the null byte for measurement name, and byte
255for the field key.Keys will now be ordered as follows:
which is preferable in terms of access patterns, and means we need two bytes less per series key.
Finally, this change will ensure that tag sets are ordered the same way in series keys on both the 1.x and 2.x lines. We can guarantee this because we will only ever be adding a tag pair to the beginning and end of a tag set. This should lead to more elegant and performant implementations of import/export of TSM data between the 1.x and 2.x lines.
Further, this PR adds another change: