Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ReadOptions.iter_start_ts should support tombstones #7178

Closed

Conversation

riversand963
Copy link
Contributor

@riversand963 riversand963 commented Jul 27, 2020

Summary: as title.
When ReadOptions.iter_start_ts is not nullptr, DBIter::key() should
return internal keys including value type.

Test Plan:
make check

@starsiriusa
Copy link

InternalKey hasn't been exposed so far. ParseInternalKey() and ParsedInternalKey are not available for application to consume.
Or there is already interface for it?

@riversand963
Copy link
Contributor Author

InternalKey hasn't been exposed so far. ParseInternalKey() and ParsedInternalKey are not available for application to consume.
Or there is already interface for it?

The C++ types InternalKey or ParsedInternalKey are not exposed. Neither is ParseInternalKey() function. However, RocksDB has been able to return the internal key as a Slice back to application. For example, if ReadOptions.iter_start_seqnum == true, then internal keys will be returned as Slice. The motivation is that some applications do want to list all versions of data including tombstones.

@starsiriusa
Copy link

I guess any application that is taking advantage of iter_start_seqnum_ is also assuming knowledge of internal key format. Not sure if it was intentional, but would be nice if support exists in the interface without internal structure being exposed.

@riversand963
Copy link
Contributor Author

@starsiriusa , actually in include/rocksdb/types.h, we have ParseFullKey(const Slice& internal_key, FullKey* fkey) exposed. FullKey can be found in rocksdb/types.h.
So the internal key is somewhat exposed via FullKey, though rocksdb/types.h may not be super up-to-date.

@hliu18
Copy link
Contributor

hliu18 commented Jul 28, 2020

Yes ParseFullKey() does the job.

@hliu18
Copy link
Contributor

hliu18 commented Jul 28, 2020

Maybe add a new type for GetEntryType(kTypeDeletionWithTimestamp)?

@riversand963
Copy link
Contributor Author

riversand963 commented Jul 28, 2020

Maybe add a new type for GetEntryType(kTypeDeletionWithTimestamp)?

SG. Will do tmr.

Update:
On second thought, I think this can be done in a separate PR.

Copy link
Contributor

@ltamasi ltamasi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @riversand963 ! LGTM, just a couple of minor comments.

for (it->Seek(Key1(0)), key = 0; it->Valid(); it->Next(), ++count, ++key) {
CheckIterEntry(it.get(), Key1(key), kTypeDeletionWithTimestamp, Slice(),
write_ts);
it->Next();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The purpose of this is to skip the Put with timestamp 3, right? If so, I think a comment saying so would be useful here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct.

CheckIterEntry(iter.get(), Key1(key), expected_seq, kTypeValue,
CheckIterEntry(iter.get(), Key1(key), expected_seq,
(key % 2) ? kTypeValue : kTypeDeletionWithTimestamp,
"value" + std::to_string(i + 1), write_ts_list[i + 1]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: we could avoid passing expected_value if it's a deletion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you suggest adding an overloaded version of CheckIterEntry that takes one fewer argument?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would also work but what I had in mind originally was to avoid creating the value string for deletes (since it's unused by the callee), i.e. something along the lines of (key % 2) ? ("value" + std::to_string(i + 1)) : std::string().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SG. Will update. Thanks!

for (size_t i = 0; i < read_ts_list.size(); ++i) {
Slice read_ts = read_ts_list[i];
Slice read_ts_lb = read_ts_lb_list[i];
(void)read_ts_lb;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: probably left over/unnecessary line?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry about this. Will remove.

Summary: as title.
When ReadOptions.iter_start_ts is not nullptr, DBIter::key() should
return internal keys including value type.

Test Plan:
make check
Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@riversand963 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@riversand963 merged this pull request in 2735b02.

@riversand963 riversand963 deleted the dbiter-return-tombstones branch August 5, 2020 03:10
codingrhythm pushed a commit to SafetyCulture/rocksdb that referenced this pull request Mar 5, 2021
Summary:
as title.
When ReadOptions.iter_start_ts is not nullptr, DBIter::key() should
return internal keys including value type.

Pull Request resolved: facebook#7178

Test Plan: make check

Reviewed By: ltamasi

Differential Revision: D22935879

Pulled By: riversand963

fbshipit-source-id: 7508d962cf11ebcfa6386d2529b4f3606b47ccfd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants