Track per-SST user-defined timestamp information in MANIFEST #8957
Assignees
Labels
Comments
|
@riversand963 ok, we'll try to privide a mr recently base on your advises, and discuss with the code. |
|
I post a merge request for this issue |
|
@riversand963 I repost a new pull request for this issue. #9092 |
sunlike-Lipo
added a commit
to sunlike-Lipo/rocksdb
that referenced
this issue
Nov 4, 2021
sunlike-Lipo
added a commit
to sunlike-Lipo/rocksdb
that referenced
this issue
Nov 8, 2021
…8957 Rockdb has supported user-defined timestamp feature. Application can specify a timestamp when writing each k-v pair. When data flush from memory to disk file called SST files, file creation activity will commit to MANIFEST. This commit is for tracking timestamp info in the MANIFEST for each file. The changes involved are as follows: 1) Track max/min timestamp in FileMetaData, and fix invoved codes. 2) Add max/min timestamp in NewFileCustomTag(in the knEWfILE4 part), and support invoved codes such as VersionEdit Encode and Decode etc. 3) Add unit test code for VersionEdit EncodeDecodeNewFile4, and fix invoved test codes.
facebook-github-bot
pushed a commit
that referenced
this issue
Nov 10, 2021
Summary: Track per-SST user-defined timestamp information in MANIFEST #8957 Rockdb has supported user-defined timestamp feature. Application can specify a timestamp when writing each k-v pair. When data flush from memory to disk file called SST files, file creation activity will commit to MANIFEST. This commit is for tracking timestamp info in the MANIFEST for each file. The changes involved are as follows: 1) Track max/min timestamp in FileMetaData, and fix invoved codes. 2) Add NewFileCustomTag::kMinTimestamp and NewFileCustomTag::kMinTimestamp in NewFileCustomTag ( in the kNewFile4 part ), and support invoved codes such as VersionEdit Encode and Decode etc. 3) Add unit test code for VersionEdit EncodeDecodeNewFile4, and fix invoved test codes. Pull Request resolved: #9092 Reviewed By: ajkr, akankshamahajan15 Differential Revision: D32252323 Pulled By: riversand963 fbshipit-source-id: d2642898d6e3ad1fef0eb866b98045408bd4e162
|
PR merged. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Internal ref: T86717089
We have been working on adding support for user-defined timestamp feature. Application can specify a timestamp when writing each key-value pair. Conceptually,
Data written will first be buffered in memory in a data structure called MemTable, after a certain amount of time when certain conditions trigger, RocksDB flushes data from MemTable to files called SST files.
After writing SST files, RocksDB commits the file creation activity to the MANIFEST by calling
LogAndApply(). We should also track timestamp information in the MANIFEST for each file.Places to explore:
NewFileCustomTag, defined in version_edit.h. Note that we require these new tags to be format-compatible.VersionEdit::AddFile(),VersionEdit::DecodeFrom(),ToString(), etc.FileMetaData.We follow Google C++ style guide (https://google.github.io/styleguide/cppguide.html).
The text was updated successfully, but these errors were encountered: