Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Overloading the driver #33

Open
pmembrey opened this Issue Jul 11, 2012 · 0 comments

Comments

Projects
None yet
1 participant

Hi,

The process that actually writes to the database can be easily over loaded. If MongoDB is busy or slow for whatever reason, the messages can back up in the process' queue to the point where it cannot recover. Even a short burst can cause this problem and although MongoDB recovers almost immediately, the driver process (due to the selective receives) never recovers to the point where it can clear out the queue resulting in the app ultimately crashing.

When this occurs, the throughput goes down from around 6,000 messages per second (and being able to handle spikes of 10,000 with no problem) to about 30 per second - far less than the number coming in, which of course ultimately makes things even slower.

If I kill the producer, the process will recover eventually but it can take hours for that to happen.

Not sure what additional information to provide but I think it's caused by a selective receive and the message queue growing too fast.

Cheers,

Pete

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment