-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enhanced bytes limiter with data type param #7414
Conversation
f910fdc
to
6b0040d
Compare
Signed-off-by: Justin Jung <jungjust@amazon.com>
Signed-off-by: Justin Jung <jungjust@amazon.com>
Signed-off-by: Justin Jung <jungjust@amazon.com>
Signed-off-by: Justin Jung <jungjust@amazon.com>
Signed-off-by: Justin Jung <jungjust@amazon.com>
Signed-off-by: Justin Jung <jungjust@amazon.com>
Signed-off-by: Justin Jung <jungjust@amazon.com>
Signed-off-by: Justin Jung <jungjust@amazon.com>
Signed-off-by: Justin Jung <jungjust@amazon.com>
Signed-off-by: Justin Jung <jungjust@amazon.com>
Signed-off-by: Justin Jung <jungjust@amazon.com>
Signed-off-by: Justin Jung <jungjust@amazon.com>
Signed-off-by: Justin Jung <jungjust@amazon.com>
74d2267
to
93136a7
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Just a small nit on the changelog
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work Justin!
Signed-off-by: Justin Jung <jungjust@amazon.com>
Maybe @fpetkovski @MichaHoffmann or @saswatamcode can help take a look? Thanks |
I will merge this change tomorrow if there is no objection or new review comments. Since this change is not user facing at all, we can address in future PRs if you have any other comments |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, generally LGTM! I think it can be put in changelog as your Store error will now tell you the type of data that exceeded bytes limit?
Also any plans to expose these to user somehow?
Thanks for taking a look @saswatamcode !
I haven't changed the error message returned to users, as they already contained the data type information. The main goal of this change was to enhance the limiter logic with access to the data type, which can be hidden from the users.
Yes! We have been noticing that resource consumption (CPU and memory) for postings, series and chunks are quite different. For example, we were able to fetch up to 1GB/s of chunks until the CPU is exhausted, while we were only able to fetch 258MB/s of series. We should definitely update the limiter logic to assign different weights per data type. |
* Refactor existing stats incrementation for touched and fetched data Signed-off-by: Justin Jung <jungjust@amazon.com> * Add TypedBytesLimiter Signed-off-by: Justin Jung <jungjust@amazon.com> * Remove addAndCheck func Signed-off-by: Justin Jung <jungjust@amazon.com> * Update BytesLimiter interface to accept dataType param Signed-off-by: Justin Jung <jungjust@amazon.com> * Added tests Signed-off-by: Justin Jung <jungjust@amazon.com> * Fix build + changelog Signed-off-by: Justin Jung <jungjust@amazon.com> * Fix wrong data type Signed-off-by: Justin Jung <jungjust@amazon.com> * Changed storeDataType to be exported Signed-off-by: Justin Jung <jungjust@amazon.com> * Revert []BytesLimiter to BytesLimtier Signed-off-by: Justin Jung <jungjust@amazon.com> * Lint Signed-off-by: Justin Jung <jungjust@amazon.com> * More reverts Signed-off-by: Justin Jung <jungjust@amazon.com> * More Signed-off-by: Justin Jung <jungjust@amazon.com> * Rename DefaultBytesLimiterFactory back to NewBytesLimiterFactory Signed-off-by: Justin Jung <jungjust@amazon.com> * Changed StoreDataType from string to int Signed-off-by: Justin Jung <jungjust@amazon.com> * Removed nil check for bytesLimiter Signed-off-by: Justin Jung <jungjust@amazon.com> * nit Signed-off-by: Justin Jung <jungjust@amazon.com> * Removed changelog Signed-off-by: Justin Jung <jungjust@amazon.com> --------- Signed-off-by: Justin Jung <jungjust@amazon.com>
* Refactor existing stats incrementation for touched and fetched data Signed-off-by: Justin Jung <jungjust@amazon.com> * Add TypedBytesLimiter Signed-off-by: Justin Jung <jungjust@amazon.com> * Remove addAndCheck func Signed-off-by: Justin Jung <jungjust@amazon.com> * Update BytesLimiter interface to accept dataType param Signed-off-by: Justin Jung <jungjust@amazon.com> * Added tests Signed-off-by: Justin Jung <jungjust@amazon.com> * Fix build + changelog Signed-off-by: Justin Jung <jungjust@amazon.com> * Fix wrong data type Signed-off-by: Justin Jung <jungjust@amazon.com> * Changed storeDataType to be exported Signed-off-by: Justin Jung <jungjust@amazon.com> * Revert []BytesLimiter to BytesLimtier Signed-off-by: Justin Jung <jungjust@amazon.com> * Lint Signed-off-by: Justin Jung <jungjust@amazon.com> * More reverts Signed-off-by: Justin Jung <jungjust@amazon.com> * More Signed-off-by: Justin Jung <jungjust@amazon.com> * Rename DefaultBytesLimiterFactory back to NewBytesLimiterFactory Signed-off-by: Justin Jung <jungjust@amazon.com> * Changed StoreDataType from string to int Signed-off-by: Justin Jung <jungjust@amazon.com> * Removed nil check for bytesLimiter Signed-off-by: Justin Jung <jungjust@amazon.com> * nit Signed-off-by: Justin Jung <jungjust@amazon.com> * Removed changelog Signed-off-by: Justin Jung <jungjust@amazon.com> --------- Signed-off-by: Justin Jung <jungjust@amazon.com>
Changes
Introducing two main changes to enhance bytes-based limiting:
BytesLimiter.Reserve(num uint64) err
toBytesLimiter.ReserveWithType(num uint64, dataType storeDataType) err
such that the bytes limiter logic to be more flexible. This is useful given that processing of postings, series and chunks have different impact to pod resource usage. For example, user can have a bytes limiter that has different weights for different data types.queryStats.add
func to update query stats info more consistentlyVerification
Added unit test and e2e test.