Write file temperature information to manifest #8284
Conversation
|
@siying has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
| kCold, | ||
| kTotal, | ||
| kUnknown = kTotal, | ||
| kUnknown = 0, |
There was a problem hiding this comment.
Should we give a larger range between kHot and kCold for future extension not just 1 to 3?
There was a problem hiding this comment.
Hmm... I don't know. I can leave some room just in case.
That's a good question. Maybe we can delay the decision a little bit? The downside of tagging a file hot is that we will need to put a field in the manifest file. Doing it for a unfinished feature feels scaring to me. Maybe we can first only tag cold files, which is necessary for development and benchmarking, and decide whether we need to tag hot files too in the future? |
Yeah, it make sense. |
db/version_edit.h
Outdated
| @@ -87,7 +87,8 @@ enum NewFileCustomTag : uint32_t { | |||
| kCustomTagNonSafeIgnoreMask = 1 << 6, | |||
|
|
|||
| // Forward incompatible (aka unignorable) fields | |||
| kPathId, | |||
| kPathId = (1 << 6) | 1, | |||
| kTemperature = (1 << 6) | 2, | |||
There was a problem hiding this comment.
I'm rethinking about it and moving it to forward compatible field.
Summary: As a part of tiered storage, writing tempeature information to manifest is needed so that after DB recovery, RocksDB still has the tiering information, to implement some further necessary functionalities. Also fix some issues in simulated hybrid FS. Test Plan: Add a new unit test to validate that the information is indeed written and read back.
|
@siying has updated the pull request. You must reimport the pull request before landing. |
|
@siying has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
|
@siying has updated the pull request. You must reimport the pull request before landing. |
|
@siying has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
|
This pull request has been merged in 0ed8cb6. |
Summary:
As a part of tiered storage, writing tempeature information to manifest is needed so that after DB recovery, RocksDB still has the tiering information, to implement some further necessary functionalities.
Also fix some issues in simulated hybrid FS.
Test Plan: Add a new unit test to validate that the information is indeed written and read back.