Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage/reader: Avoid some copy in mvcc reader #4373

Merged
merged 3 commits into from Mar 15, 2019

Conversation

MyonKeminta
Copy link
Contributor

Signed-off-by: MyonKeminta MyonKeminta@users.noreply.github.com

What have you changed? (mandatory)

This PR tries to avoid some clones of keys in mvcc reader.

What are the type of the changes? (mandatory)

  • Improvement (change which is an improvement to an existing feature)

How has this PR been tested? (mandatory)

By unit tests

Does this PR affect documentation (docs) update? (mandatory)

No

Does this PR affect tidb-ansible update? (mandatory)

No

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
@MyonKeminta
Copy link
Contributor Author

@breeswish @Connor1996 PTAL!

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Copy link
Member

@Connor1996 Connor1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@breezewish
Copy link
Member

breezewish commented Mar 14, 2019

@MyonKeminta Do you have some interests to work on provide references (instead of owned values) in Mvcc Scanners? This can further eliminate copies. According to my benchmarks there can be 7% improvements in the integration benchmark (using a MySQL client)

@MyonKeminta
Copy link
Contributor Author

@breeswish Of course! But I think I need some time to think about how to do that 🤔

@breezewish
Copy link
Member

@MyonKeminta I once had an experiment over this. The code is pretty awful but the outcome is great. The main challenge (of writing elegant code) is that currently we use Key structure, which enforces type safety of encoded values or raw values, which is great, but itself only support owned values. There is no way to enforce type safety by only providing references. For example, our Cursor::seek and other friends accept &Key, which requires an owned Vec<..> being built (and then pass it as a reference). This fundamentally make it impossible to be zero copy (because what we receive from RocksDB is just a &[u8] ).

@breezewish
Copy link
Member

Previous dirty attempt: breezewish@96afbb4

@siddontang
Copy link
Contributor

Thanks @morefreeze
Have you done some bechmarks about this?

@Connor1996 Connor1996 added status/LGT2 Status: PR - There are already 2 approvals sig/transaction SIG: Transaction component/performance Component: Performance labels Mar 15, 2019
@Connor1996 Connor1996 merged commit 01ca726 into tikv:master Mar 15, 2019
@morefreeze
Copy link
Contributor

Sorry, why this involves me? I have no idea about these changes. Is that a misspell?

@breezewish
Copy link
Member

@morefreeze Yes it should be a misspell, nevermind :)

@MyonKeminta MyonKeminta deleted the misono/avoid-a-copy branch March 15, 2019 05:56
@MyonKeminta
Copy link
Contributor Author

@siddontang Not yet. I didn't expect this PR to significantly improve the performance, but I just thought that it's never bad to remove the unnecessary copies.
@breeswish Thank you! I'll check it later. By the way, I'm considering whether we can let the Key able to represent both owned value and references by using unsafe code... 🤔

@breezewish
Copy link
Member

@MyonKeminta How about just KeyRef for references and Key for owned values? (I don't know. just a brain storm)

@MyonKeminta
Copy link
Contributor Author

@breeswish I don't know either before a try 😂

sticnarf pushed a commit to sticnarf/tikv that referenced this pull request Oct 27, 2019
* Avoid some copy in mvcc reader

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/performance Component: Performance sig/transaction SIG: Transaction status/LGT2 Status: PR - There are already 2 approvals
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants