fix(core-p2p): disable permessage-deflate #3518
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
alessiodf
requested review from
air1one,
faustbrian and
kristjank
as code owners
|
Thanks for submitting this pull request! A maintainer will review this in the next few days and explicitly select labels so you know what's going on. If no reviewer appears after a week, a reminder will be sent out. |
faustbrian
approved these changes
Feb 20, 2020
|
A collaborator has approved this PR. A maintainer will merge this PR shortly. If it shouldn't be merged yet, please leave a comment saying so and we'll wait. Thank you for your contribution! |
|
Your pull request has been merged and marked as tier 1. It will earn you $200 USD. |
ghost
mentioned this pull request
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
There is memory leak in
core-p2pwhich also leads to workers not responding after a while. This is caused by enablingpermessage-deflate(PMD) in the websockets. PMD was introduced in 2.5 but the problem only became apparent in 2.6 because PMD uses heuristics to determine whether to compress the data or send it uncompressed, for example, small packets are not compressed because the overhead would not be worth it. AIP11 introduced things like multipayments, which result in larger transaction payloads, triggering PMD compression whereas the smaller transaction payloads in Core 2.5 were below the threshold for PMD compression. This is why the problem only became apparent in 2.6 and also likely explains why 2.6 uses more CPU cycles as PMD is happening much more often, which requires more resources to compress/uncompress the data.Due to the current p2p architecture, using PMD means the websocket can never be garbage collected (GC). Over time the number of websockets keeps rising and this causes a memory leak. Eventually, the leak stops the PMD compression queues from consuming their data, so any data to be compressed does not get sent to the destination websocket. This is what causes the workers to appear to stop responding, since their compressed output never gets consumed in the compression queue. It also explains why the
p2p.peer.postTransactionsendpoint still works -- it only returns an empty array, below the PMD compression threshold, whereas other endpoints return more data which would be compressed.Due to the new optimised data transmission codec and serialisation, PMD is no longer required and will make for a leaner less resource-hungry Core.
Checklist