Skip to content

Conversation

@mergify
Copy link

@mergify mergify bot commented Feb 19, 2025

Proposed Changes

Since 4.0.0 (commit d45fbc3) the shared message store writes large messages into their own rdq files. This information can be utilised when scanning rdq files during recovery to avoid reading in the whole message body into memory unnecessarily.

This commit addresses the same issue that was addressed in 3.13.x by commit baeefbe (ie. appending a large binary together from 4MB chunks leaves a lot of garbage and memory fragmentation behind) but even more efficiently.

Large messages which were written before 4.0.0, which don't fully fill the rdq file, are still handled as before.

Types of Changes

What types of changes does your code introduce to this project?
Put an x in the boxes that apply

  • Bug fix (non-breaking change which fixes issue #NNNN)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause an observable behavior change in existing systems)
  • Documentation improvements (corrections, new content, etc)
  • Cosmetic change (whitespace, formatting, etc)
  • Build system and/or CI

Checklist

Put an x in the boxes that apply.
You can also fill these out after creating the PR.
If you're unsure about any of them, don't hesitate to ask on the mailing list.
We're here to help!
This is simply a reminder of what we are going to look for before merging your code.

  • I have read the CONTRIBUTING.md document
  • I have signed the CA (see https://cla.pivotal.io/sign/rabbitmq)
  • I have added tests that prove my fix is effective or that my feature works
  • All tests pass locally with my changes
  • If relevant, I have added necessary documentation to https://github.com/rabbitmq/rabbitmq-website
  • If relevant, I have added this change to the first version(s) in release-notes that I expect to introduce it

Further Comments

If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution
you did and what alternatives you considered, etc.


This is an automatic backport of pull request #13250 done by [Mergify](https://mergify.com).

Since 4.0.0 (commit d45fbc3) the shared message store writes large
messages into their own rdq files. This information can be utilised
when scanning rdq files during recovery to avoid reading in the whole
message body into memory unnecessarily.

This commit addresses the same issue that was addressed in 3.13.x by
commit baeefbe (ie. appending a large binary together from 4MB chunks
leaves a lot of garbage and memory fragmentation behind) but even more
efficiently.

Large messages which were written before 4.0.0, which don't fully fill
the rdq file, are still handled as before.

(cherry picked from commit fb21a19)
@michaelklishin michaelklishin merged commit 9b9a261 into v4.1.x Feb 20, 2025
270 checks passed
@michaelklishin michaelklishin deleted the mergify/bp/v4.1.x/pr-13250 branch February 20, 2025 00:18
michaelklishin added a commit that referenced this pull request Feb 20, 2025
4.x: Optimise msg_store recovery in case of large message file (backport #13250) (backport #13373)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants