Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid parsing large sizes for messages #1441

Merged
merged 1 commit into from
Jun 2, 2020
Merged

Avoid parsing large sizes for messages #1441

merged 1 commit into from
Jun 2, 2020

Conversation

derekcollison
Copy link
Member

Copy link
Member

@kozlovic kozlovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is the correct approach..

l := len(d)
if l == 0 {
if l == 0 || l > maxParseSizeLen {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although I agree that NATS users should not send large messages, you are limiting it to about 99MB. Not sure if this is going to break users that have bumped the max payload.
I think that the new issue reported is simply that the server was trying to allocate more memory that the machine had, not that there was a rollover to negative that was not handled properly in the past.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the original fix: #1053

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have max pending buffers internal that are 64M so I thought that was reasonable.

Signed-off-by: Derek Collison <derek@nats.io>
Copy link
Member

@kozlovic kozlovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. As discussed, servers that are deployed and have a max_payload set (by default it is 1MB) are not impacted by this PUB foo 333333333333333333\r\n protocol. If no limit is set (or set to very high value), then this change will apply a hard limit of a size of 9 digits. It is still possible that the server will produce the stack because it would fail to allocate memory (the make([]byte) would panic due to lack of memory). Ultimately, we should warn people who set a big/unlimited max_payload and in the future even error if the value is more than a certain size (which one is tbd).

@derekcollison derekcollison merged commit d6e7210 into master Jun 2, 2020
@derekcollison derekcollison deleted the fuzz branch June 2, 2020 00:04
@AdamKorcz AdamKorcz mentioned this pull request Jun 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants