Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[memprof] Omit the key/data lengths for the frame table #89711

Closed

Conversation

kazutakahirata
Copy link
Contributor

The frame table has constant key/data lengths, so we don't need to
serialize or deserialize them for every key-data pair. Omitting the
key/data lengths saves 0.21% of the indexed MemProf file size.

Note that it's OK to change the format because Version2 is still under
development.

The frame table has constant key/data lengths, so we don't need to
serialize or deserialize them for every key-data pair.  Omitting the
key/data lengths saves 0.21% of the indexed MemProf file size.

Note that it's OK to change the format because Version2 is still under
development.
OnDiskChainedHashTableGenerator<memprof::FrameWriterTrait>
FrameTableGenerator;
for (auto &[FrameId, Frame] : MemProfFrameData) {
// Insert the key (frame id) and value (frame contents).
FrameTableGenerator.insert(FrameId, Frame);
FrameTableGenerator.insert(FrameId, Frame, FrameWriter);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there changes needed in this patch to the insert() and Emit() methods of OnDiskChainedHashTableGenerator to take the writer trait?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there changes needed in this patch to the insert() and Emit() methods of OnDiskChainedHashTableGenerator to take the writer trait?

Yes, FrameWriter carries the version, so we need to pass FrameWriter to both insert and Emit.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I think that part is missing from the PR?

@kazutakahirata
Copy link
Contributor Author

I'm going to discard this PR. We would need to route the writer trait to even more places (than present in this patch) to correctly invoke Info.ReadKeyDataLength() (yes, with Info, not as the static method). The impact on the file size is very small for the number of places we would need to touch.

@teresajohnson Thank you for reviewing this patch!

@kazutakahirata kazutakahirata deleted the pr_memprof_omit_key_frame branch May 16, 2024 07:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants