Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce memory allocation in handlerDeliver #294

Merged
merged 3 commits into from
Apr 16, 2024
Merged

Reduce memory allocation in handlerDeliver #294

merged 3 commits into from
Apr 16, 2024

Conversation

hiimjako
Copy link
Contributor

As discussed in #293, I've removed some unnecessary memory allocations during the parsing of a message. This improvement results in higher msg/s.

Gsantomaggio
Gsantomaggio approved these changes Apr 16, 2024
View reviewed changes
Copy link
Member

@Gsantomaggio Gsantomaggio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you

@Gsantomaggio
Copy link
Member

This improvement results in higher msg/s.

Out of curiosity: Do you have some numbers regarding performance gain?

@Gsantomaggio Gsantomaggio self-assigned this Apr 16, 2024
@Gsantomaggio Gsantomaggio added this to the 1.4.1 milestone Apr 16, 2024
@Gsantomaggio Gsantomaggio merged commit 2376300 into rabbitmq:main Apr 16, 2024
2 checks passed
@hiimjako
Copy link
Contributor Author

hiimjako commented Apr 16, 2024 •
edited

On the main branch

go run perfTest/perftest.go --publishers=0 --consumers=1 --streams=measurement --consumer-offset=first --max-length-bytes=20GB --time=5
2024/04/16 09:54:58 [info] - Silent (1.4.0) Simulation, url: [rabbitmq-stream://guest:guest@localhost:5552/%2f] publishers: 0 consumers: 1 streams: [measurement] 
2024/04/16 09:54:58 [info] - Declaring streams: [measurement]
2024/04/16 09:54:58 [info] - stream measurement, meta data: leader localhost:5552, followers  
2024/04/16 09:54:58 [info] - End Init streams :[measurement]
2024/04/16 09:54:58 [info] - Starting 1 consumers...
2024/04/16 09:54:58 [info] - Starting consumer number: measurement-0, form first
2024/04/16 09:55:00 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 729056.0 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:55:01 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 737264.0 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:55:02 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 729077.3 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:55:03 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 724984.0 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:55:04 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 717612.8 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:55:05 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 709849.7 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:55:06 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 711529.1 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:55:07 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 715738.4 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:55:08 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 716341.3 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:55:09 [info] - Stopping after 5 seconds

on this branch

go run perfTest/perftest.go --publishers=0 --consumers=1 --streams=measurement --consumer-offset=first --max-length-bytes=20GB --time=5
2024/04/16 09:54:05 [info] - Silent (1.4.0) Simulation, url: [rabbitmq-stream://guest:guest@localhost:5552/%2f] publishers: 0 consumers: 1 streams: [measurement] 
2024/04/16 09:54:05 [info] - Declaring streams: [measurement]
2024/04/16 09:54:05 [info] - stream measurement, meta data: leader localhost:5552, followers  
2024/04/16 09:54:05 [info] - End Init streams :[measurement]
2024/04/16 09:54:05 [info] - Starting 1 consumers...
2024/04/16 09:54:05 [info] - Starting consumer number: measurement-0, form first
2024/04/16 09:54:06 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 859107.0 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:54:07 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 856048.0 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:54:08 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 871072.0 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:54:09 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 870588.0 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:54:10 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 863430.4 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:54:11 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 876392.6 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:54:12 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 883561.1 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:54:13 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 883708.0 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:54:14 [info] - Published      0.0 msg/s | Confirmed      0.0 msg/s |  Consumed 882912.0 msg/s |   ND  |   ND  |  msg sent:   0  |
2024/04/16 09:54:15 [info] - Stopping after 5 seconds

So from like ~720k messages to ~870k, so it is like ~20%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Gsantomaggio Gsantomaggio Gsantomaggio approved these changes

Assignees

@Gsantomaggio Gsantomaggio

Labels
None yet
Projects
None yet
Milestone
1.4.1
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@hiimjako @Gsantomaggio