Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adopt distributed version Bucket4j for huge request rate on single key #90

Closed
vladimir-bukhtoyarov opened this issue Jan 11, 2019 · 2 comments
Closed

Adopt distributed version Bucket4j for huge request rate on single key #90

vladimir-bukhtoyarov opened this issue Jan 11, 2019 · 2 comments
Labels
feature
Projects
7.0 - Conquer the world

Comments

@vladimir-bukhtoyarov
Copy link
Collaborator

vladimir-bukhtoyarov commented Jan 11, 2019 •
edited

Problem:
Because Bucket4j makes a new network request to backend on each limitation check the Bucket4j it is not well suitable for cases when the request rate is huge(tens thouthands per second), because remote backend becomes become a bottleneck.

Solution:
Hold bucket state locally on each node and sync consumed tokens periodically with a configured interval. Be aware of consumption rate inside cluster.

Ideas for implementation:
The algorithms are well described in research articles for example:
Cloud Control with Distributed Rate Limiting Barath Raghavan, Kashi Vishwanath, Sriram Ramabhadran, Kenneth Yocum, and Alex C. Snoeren. Department of Computer Science and Engineering University of California, San Diego

Limitations:
Acting in this way may break consistency, so activation of this mode should be optional.
vladimir-bukhtoyarov added a commit that referenced this issue Sep 22, 2019
@vladimir-bukhtoyarov
#90 API refactoring
edaefb9
vladimir-bukhtoyarov added a commit that referenced this issue Sep 22, 2019
@vladimir-bukhtoyarov
#90 API refactoring
bc41365
vladimir-bukhtoyarov pushed a commit that referenced this issue Oct 2, 2019
#90 API refactoring
c5a1b08
vladimir-bukhtoyarov pushed a commit that referenced this issue Oct 7, 2019
#90 API refactoring
a01eb19
vladimir-bukhtoyarov pushed a commit that referenced this issue Oct 7, 2019
#90 API refactoring
b370802
vladimir-bukhtoyarov pushed a commit that referenced this issue Oct 16, 2019
#90 API refactoring
2bbff1d
vladimir-bukhtoyarov pushed a commit that referenced this issue Oct 17, 2019
#90 API refactoring
706c617
vladimir-bukhtoyarov pushed a commit that referenced this issue Oct 17, 2019
#90 API refactoring
a305354
vladimir-bukhtoyarov pushed a commit that referenced this issue Oct 17, 2019
#90 API refactoring
ccf5f8f
vladimir-bukhtoyarov pushed a commit that referenced this issue Oct 18, 2019
#90 API refactoring
396f955
vladimir-bukhtoyarov pushed a commit that referenced this issue Oct 18, 2019
#90 API refactoring
6686cb4
@vladimir-bukhtoyarov vladimir-bukhtoyarov changed the title Adopt distributed version Bucket4j for huge request rate Adopt distributed version Bucket4j for huge request rate on single key Oct 27, 2019
vladimir-bukhtoyarov added a commit that referenced this issue Oct 27, 2019
@vladimir-bukhtoyarov
#90 implement batching optimization for case of huge rate on the same…
6dce4ac 
… distributed bucket
vladimir-bukhtoyarov added a commit that referenced this issue Oct 27, 2019
@vladimir-bukhtoyarov
#90 add example which shows 150000 request/second throughput on singl…
509d55c 
…e Hazelcast bucket with 100 concurrent threads
vladimir-bukhtoyarov pushed a commit that referenced this issue Oct 28, 2019
#90 Batching optimization strategy for AsyncBucket
540ab18
vladimir-bukhtoyarov pushed a commit that referenced this issue Nov 12, 2019
#90 example which describes how to achieve approximately 1 millions r…
d6766c2 
…equests per second with asynchronous buckets and Hazelcast
vladimir-bukhtoyarov pushed a commit that referenced this issue Jan 9, 2020
#90 example which describes how to achieve approximately 1 millions r…
6415414 
…equests per second with asynchronous buckets and Hazelcast
vladimir-bukhtoyarov pushed a commit that referenced this issue Jan 10, 2020
#90 make decomposition of batching executor in order to achieve bette…
4cbc154 
…r unit testing
vladimir-bukhtoyarov pushed a commit that referenced this issue Jan 10, 2020
@vladimir-bukhtoyarov
#90 make decomposition of batching executor in order to achieve bette…
ba714de 
…r unit testing
vladimir-bukhtoyarov added a commit that referenced this issue Jan 10, 2020
@vladimir-bukhtoyarov
#90 replace synchronization blocks by lock-free structures
9150817
@vladimir-bukhtoyarov vladimir-bukhtoyarov moved this from In-progress to Implemented in 7.0 - Conquer the world Mar 7, 2020
vladimir-bukhtoyarov added a commit that referenced this issue Jul 28, 2020
@vladimir-bukhtoyarov
#90 predictive optimizer
ce4faf7
vladimir-bukhtoyarov added a commit that referenced this issue Jul 28, 2020
@vladimir-bukhtoyarov
#90 predictive optimizer
1d34466
vladimir-bukhtoyarov added a commit that referenced this issue Jul 29, 2020
@vladimir-bukhtoyarov
#90 predictive optimizer
ea8774a
vladimir-bukhtoyarov added a commit that referenced this issue Jul 30, 2020
@vladimir-bukhtoyarov
#90 predictive optimizer
3fab786
vladimir-bukhtoyarov added a commit that referenced this issue Jul 30, 2020
@vladimir-bukhtoyarov
#90 delayed optimization
681eae4
vladimir-bukhtoyarov added a commit that referenced this issue Jul 30, 2020
@vladimir-bukhtoyarov
#90 monitoring API for optimizations
0a8f5a6
vladimir-bukhtoyarov added a commit that referenced this issue Jul 31, 2020
@vladimir-bukhtoyarov
#90 tests + bugfixes for delaying consumption optimization
78f49d4
vladimir-bukhtoyarov added a commit that referenced this issue Aug 1, 2020
@vladimir-bukhtoyarov
#90 tests + bugfixes for delaying consumption optimization
5f18767
vladimir-bukhtoyarov added a commit that referenced this issue Aug 7, 2020
@vladimir-bukhtoyarov
#90 predictive optimizer - unit tests + bugfixes
7e30ba5
@vladimir-bukhtoyarov
Copy link
Collaborator Author

Implementation has been done, documentation will be created in the scope of #69

vladimir-bukhtoyarov added a commit that referenced this issue Nov 2, 2020
@vladimir-bukhtoyarov
#90 extract batching to dedicated abstraction in order to simplify un…
c72e802 
…it-tests for concurrent scenarios
@vladimir-bukhtoyarov vladimir-bukhtoyarov moved this from Implemented to Documented in 7.0 - Conquer the world Feb 10, 2022
@vladimir-bukhtoyarov vladimir-bukhtoyarov moved this from Documented to Implemented in 7.0 - Conquer the world Feb 10, 2022
@rupebac

This comment was marked as abuse.

Sign in to view
@vladimir-bukhtoyarov vladimir-bukhtoyarov moved this from Implemented to Documented in 7.0 - Conquer the world Sep 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature
Projects
No open projects
7.0 - Conquer the world
Documented
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rupebac @vladimir-bukhtoyarov