-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
StepExecution counts integer overflow #3650
Comments
Also noted, apply method which increments counts is synchronized. Should we switch from volatile to atomic long? |
Similar/Related to #1055. |
@benas any update on this? we are getting integer overflow :) |
@benas, If this is up for grabs, I would like to take a shot at it. It would be helpful if you could provide some pointers. |
@ArtyomGabeev Thank you for reporting this issue. The problem can also happen in a locally or remotely partitioned/chunked step: each worker could contribute a read/write count and the aggregation might silently fail with an integer overflow on the manager side, leading to incorrect counters in the aggregate Changing the type of metrics from |
Hi,
In StepExecution class counts like readCount/writeCount, etc has int type.
In case when step operates on a big amount of items, there may be an int overflow and may affect e.g. progress monitoring if someone use these counters.
Should we change it to long instead?
The text was updated successfully, but these errors were encountered: