Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed #573 ByteBuf reference counting #600

Merged

Conversation

hylkevds
Copy link
Collaborator

@hylkevds hylkevds commented May 24, 2021

The reworks most of the ByteBuf reference counting by moving all buffer release/retain calls to the outside interfaces of moquette.
This supersedes #576 and is based on the discussion there.

Close #576
Close #599
Fix #573

Moved buffer release/retain calls to the outside interfaces of moquette.
Copy link
Collaborator

@andsel andsel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andsel
Copy link
Collaborator

andsel commented Jun 5, 2021

Hi @hylkevds I was working on master creating a unit test that exposes any buffer leak in Netty, but was unfruitful. My test are on #602 could you share how do you test it?

@hylkevds
Copy link
Collaborator Author

hylkevds commented Jun 5, 2021

Testing for memory leaks with unit tests is generally very difficult, even if you know exactly what you're looking for.
Most of the time I just use small program that pushes a lot of messages in batches through several clients that regularly subscribe, unsubscribe and reconnect. You can find that here: https://github.com/hylkevds/moquetteTests/

Some characteristics of the load generator that tend to expose problems:

  • Multiple clients on the same topic
  • Clients that unsubscribe and re-subscribe
  • Clients that disconnect and re-connect
  • publishers that put out messages in batches. This ensures that buffers fill up, and then have a chance to empty again

The load generator keeps track of counts of sent and received messages, one per second. The sent/receive counts never really match exactly when clients unsubscribe and resubscribe, but they do give a quick overview if things are still running as expected. That is for instance how I found the deadlock problem with QOS2 (#587)

I just run it in the profiler in Netbeans, so I can track memory usage and object generations. If memory usage increases significantly over an hour or so, you know there is an issue...
For shorter tests, setting the Netty leak detection to paranoid also helps.

@andsel andsel merged commit 7472222 into moquette-io:master Jun 6, 2021
@hylkevds hylkevds deleted the fix_573_2_ByteBufRefCount branch July 2, 2021 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0.14: IllegalReferenceCountException: refCnt: 0
2 participants