Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adapt to the new IDeltaSupport API to save a copy during persistence #73

Merged
merged 3 commits into from
May 9, 2024

Conversation

songweijia
Copy link
Contributor

@songweijia songweijia commented Apr 30, 2024

I've regression-tested this implementation. It works correctly. The main change is shifting from the old finalizeDelta API to currentDeltaToBytes/currentDeltaSize API.

  • Now, the running delta in DeltaCascadeStoreCore is a vector of type std::vector<KT> for the keys that are touched in the current version. No need to do deserialization before writing to log like we did.
  • We saved a copy during persisting a log: the latest memory state is directly serialized to the memory-mapped log file. Before this change, the new memory state is serialized to the delta buffer, and then copied from the delta buffer to log file. We should do persistence evaluation again to validate the performance improvement.
  • Redesigned/implemented Cascade Delta format. I introduced a new class called DeltaCascadeStoreCore<KT,VT,IK,IV>::DeltaType as a view window for a delta log entry, which contains a map from keys to values. Unlike the normal ByteRepresentable objects, this type is only for deserialization--it only provide an approach for reading the serialized delta data but not for serialization. Serialization is done in the implementation of currentDeltaToBytes() without using a concrete type.

The action build check failed because this pull request relies on a new feature to be merged into derecho. Don't mind it.

@etremel @tgarr Could you help review this change, because it might affect the projects you are working on?

…ew for the serialized log; bugfixing currentDeltaToBytes() which should clear the delta.

TODO: smoke test and regression test.
@songweijia songweijia merged commit d214200 into master May 9, 2024
1 of 2 checks passed
@songweijia songweijia deleted the use_zerocopy_delta_api branch July 3, 2024 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant