Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only use 48 bits for encoding timestamps and 32 bits for encoding thread IDs in RawEvent in order to make it smaller. #86

Merged
merged 2 commits into from Nov 15, 2019

Conversation

@michaelwoerister
Copy link
Contributor

michaelwoerister commented Nov 14, 2019

This restricts us to timestamps of at most ~78 hours after process start which seems acceptable.
The PR reduces the size of RawEvent (and thus the size of our event stream files) by 25%.

@michaelwoerister

This comment has been minimized.

Copy link
Contributor Author

michaelwoerister commented Nov 14, 2019

Note to myself for tomorrow: Make event encoding endianess independent as part of this PR.

@michaelwoerister michaelwoerister force-pushed the michaelwoerister:smaller-raw-event branch from 4a0a5ff to 01766c5 Nov 15, 2019
@michaelwoerister

This comment has been minimized.

Copy link
Contributor Author

michaelwoerister commented Nov 15, 2019

I added two commits:

  • Store events properly in little endian format on disk.
  • Optimize the FileSerializationSink by directly copying data to the writing buffer.

The second commit makes the FileSerializationSink exactly as fast as the MmapSerializationSink in the benchmarks we have. But I only tested on Linux and I also remember that the benchmarks were no good indication of performance when used in rustc. So maybe that commit should be split out into a separate PR.

@michaelwoerister michaelwoerister force-pushed the michaelwoerister:smaller-raw-event branch from 01766c5 to c8c1e95 Nov 15, 2019
@michaelwoerister

This comment has been minimized.

Copy link
Contributor Author

michaelwoerister commented Nov 15, 2019

I went ahead and moved the last commit to a separate, less urgent PR (#88). This PR is ready for review.

Copy link
Member

wesleywiser left a comment

Great work!

@wesleywiser wesleywiser merged commit 6764d49 into rust-lang:master Nov 15, 2019
1 check passed
1 check passed
Travis CI - Pull Request Build Passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.