Skip to content

Conversation

@atn34
Copy link
Collaborator

@atn34 atn34 commented May 3, 2020

Change the serialization for VectorRef<KeyValueRef, VecSerStrategy::String> (used by GetKeyValuesReply), to do some simple prefix compression. The max key size allowed fits in a uint16_t, so replace the uint32_t we're currently using for key size with a uint16_t. Use the remaining uint16_t to indicate the length of the prefix borrowed from the previous key. The encoding for a key-value pair now looks like this:

2byte length of common key prefix
2byte length of key suffix
... key suffix ...
4byte length of value
... value ...

If the common prefix length is always zero, then messages are the same size as before.

Calculate the length of the common prefix using a linear scan. On the storage server, for every common byte we scan, we save a memcpy for that byte so I think it ends up being a wash. In the end we're just left with smaller network messages. The client does now need to do some extra memcpy'ing to restore the prefix.

@atn34
Copy link
Collaborator Author

atn34 commented May 5, 2020

This doesn't seem to help measurably. Maybe we can revisit this.

@atn34 atn34 closed this May 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant