Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement an abstraction that opens file for auditing multiple times, once for every auditing rayon thread #2151

Merged
merged 1 commit into from
Oct 24, 2023

Conversation

nazar-pc
Copy link
Member

This seems to be a tiny bit faster on Linux, but more importantly, it is faster than even memory-mapped I/O on Windows.

Specifically numbers I shared on Rust Internals are:

  • 936 ms for single file with seek_read (+handling of partial reads identically to read_exact_at in std)
  • 245 ms for single file with memory-mapped I/O
  • 190 ms with file openes many times, once for each thead in thread pool using seek_read

Will ask come community members to bench it as well.

Code contributor checklist:

@nazar-pc nazar-pc added this pull request to the merge queue Oct 24, 2023
Merged via the queue into main with commit b1b4f9e Oct 24, 2023
11 of 23 checks passed
@nazar-pc nazar-pc deleted the auditing-multiple-open-files branch October 24, 2023 01:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants