New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CometD connect calls are causing OOM #794

Open
sunilchauhan89 opened this Issue Oct 1, 2018 · 1 comment

Comments

Projects
None yet
2 participants
@sunilchauhan89

sunilchauhan89 commented Oct 1, 2018

We are upgrading cometd server jars from 2.3.1 to 3.1.5 . We didn't upgrade client-side javascript files.

After upgrading, few cometd connect calls started taking more than a minute to respond and responded without any data. After enabling cometd DEBUG logs, we realized that when the application returns a large amount of data ( usually more 500KBs ) in the response of connect calls, connect calls don't go to AsyncJSONTransport.State.COMPLETE and it is leading to memory leak and causing the application to go out of memory.

Any idea, how this issue can be fixed?

@sbordet

This comment has been minimized.

Show comment
Hide comment
@sbordet

sbordet Oct 1, 2018

Member

We didn't upgrade client-side javascript files.

You should, it's a major release change.

As for the rest, there is too little detail to understand what's going on. 500 KiB of messages is not too much and especially if you're using AsyncJSONTransport they should not be a problem, provided that the client can read the data that the server sent.
Are you using the AcknowledgedMessagesExtension?

Like often happens, I think you have a problem in the client, that cannot keep up with the speed the server is sending data. The server will start sending data, the client will consume it too slowly, the TCP connection becomes congested and eventually the client is too overwhelmed and the connection times out.

I doubt CometD is causing any memory leak, it's been tested in very hard conditions (I'm guessing far harder than yours) and runs for months without issues.

If you have a memory leak, take a heap dump and analyze it.

In summary, I think your problems are within your application, not CometD.

If you have evidence that the problem is in CometD, please attach the evidence here - but look hard in your application first.

Member

sbordet commented Oct 1, 2018

We didn't upgrade client-side javascript files.

You should, it's a major release change.

As for the rest, there is too little detail to understand what's going on. 500 KiB of messages is not too much and especially if you're using AsyncJSONTransport they should not be a problem, provided that the client can read the data that the server sent.
Are you using the AcknowledgedMessagesExtension?

Like often happens, I think you have a problem in the client, that cannot keep up with the speed the server is sending data. The server will start sending data, the client will consume it too slowly, the TCP connection becomes congested and eventually the client is too overwhelmed and the connection times out.

I doubt CometD is causing any memory leak, it's been tested in very hard conditions (I'm guessing far harder than yours) and runs for months without issues.

If you have a memory leak, take a heap dump and analyze it.

In summary, I think your problems are within your application, not CometD.

If you have evidence that the problem is in CometD, please attach the evidence here - but look hard in your application first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment