Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use unsynchronized buffered OutputStream for history #2127

Merged
merged 3 commits into from
Mar 6, 2023

Conversation

SirYwell
Copy link
Member

@SirYwell SirYwell commented Mar 3, 2023

Overview

Description

BufferedOutputStream has a fair amount of overhead as it's thread safe. However, we don't require thread safety here, so we can use an alternative implementation from fastutil which performs better. In my tests, the time spent writing history is halved.

Submitter Checklist

  • Make sure you are opening from a topic branch (/feature/fix/docs/ branch (right side)) and not your main branch.
  • Ensure that the pull request title represents the desired changelog entry.
  • New public fields and methods are annotated with @since TODO.
  • I read and followed the contribution guidelines.

@SirYwell SirYwell requested a review from a team as a code owner March 3, 2023 14:07
@dordsor21
Copy link
Member

It's probably worth adding a note somewhere that it's changed to a non-thread-safe implementation

@SirYwell
Copy link
Member Author

SirYwell commented Mar 3, 2023

It's probably worth adding a note somewhere that it's changed to a non-thread-safe implementation

Where would you think is the best place for such comment? I was thinking about the processSet method in AbstractChangeSet as that's the point that synchronizes atm. However, the whole design isn't thread safe, writing to the stream from different threads would have caused issues even before...

@dordsor21
Copy link
Member

Only because it's a public method in a util class, so probably a javadoc note on getCompressedOS

@SirYwell
Copy link
Member Author

SirYwell commented Mar 5, 2023

I saw that DataOutputStream#write is synchronized. Overriding and reimplementing that method brings even more improvements.

Overall, with send-before-history: false this brings down a //set stone on a region (0, 0, 0) -> (1023, 319, 1023) from >70s to <20s.

Because flamegraphs are cool, here a diff comparing the baseline to the changes on this branch (green: in baseline but not in patch, red: in patch but not in baseline. Unrelated calls are hidden)
flamegraph

@SirYwell SirYwell changed the title Use unsychronized buffered OutputStream for history Use unsynchronized buffered OutputStream for history Mar 5, 2023
@NotMyFault NotMyFault added Performance Enhancement New feature or request and removed Performance labels Mar 6, 2023
@NotMyFault NotMyFault merged commit 8a30526 into main Mar 6, 2023
@NotMyFault NotMyFault deleted the perf/bos-history branch March 6, 2023 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants