Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configurable back-pressure strategy #8533

Closed
jerrinot opened this issue Jul 15, 2016 · 1 comment

Comments

Projects
None yet
2 participants
@jerrinot
Copy link
Contributor

commented Jul 15, 2016

When back-pressure is enabled:

  • Members always use back-off.
  • Clients always throws HazelcastOverloadedException

This is inconsistent. Behaviour should be configurable on both members and clients.

@pveentjer

This comment has been minimized.

Copy link
Member

commented Apr 19, 2017

If the client gets overloaded with requests, instead of applying back pressure out of the box, an OverloadException is thrown.

This is a very unintuitive exception to deal with and gives a bad out of the box experience since one now needs to litter HZ interaction with overload handling code.

The intuitive way to deal with this is not immediately expose this exception to the end user but apply some back-pressure using e.g. randomized exponential backoff (so delays). Only when the delays have exceeded some time threshold, the exception is thrown to the user. This is the way the server deals with it.

Using an overload policy a user can also realize the current behavior where an OverloadException is thrown at the moment it happens and he can add backoff logic at every HZ interaction himself; although I question if this the most practical approach.

Apart from an painful API , these OverloadException are pretty expensive due to stacktraces being created. So if back pressure isn't handled correctly, the client could be bogged down by gc. And although it might provide some back pressure, I doubt this is desired behavior.

jerrinot added a commit to jerrinot/hazelcast that referenced this issue Apr 19, 2017

Implement exponential backoff for client backpressure
Fixes hazelcast#8533
The default behaviour is still fail-fast. This is something we should
change in Hazelcast 4.

jerrinot added a commit to jerrinot/hazelcast that referenced this issue Apr 19, 2017

Implement exponential backoff for client backpressure
Fixes hazelcast#8533
The default behaviour is still fail-fast. This is something we should
change in Hazelcast 4.

jerrinot added a commit to jerrinot/hazelcast that referenced this issue May 3, 2017

Implement exponential backoff for client backpressure
Fixes hazelcast#8533
The default behaviour is still fail-fast. This is something we should
change in Hazelcast 4.

jerrinot added a commit to jerrinot/hazelcast that referenced this issue May 26, 2017

Implement exponential backoff for client backpressure
Fixes hazelcast#8533
The default behaviour is still fail-fast. This is something we should
change in Hazelcast 4.

jerrinot added a commit to jerrinot/hazelcast that referenced this issue May 29, 2017

Implement exponential backoff for client backpressure
Fixes hazelcast#8533
The default behaviour is still fail-fast. This is something we should
change in Hazelcast 4.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.