-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add synchronized around getCount dist usage #114
base: master
Are you sure you want to change the base?
Conversation
Since TDigest objects are not thread safe, we need to protect reads from getting corrupted data or potentially throwing exceptions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I may have lost track of thing. Why do we need synchronized around distRef ? It is an atomic object .
Without this change, it is possible for one thread to read the size of the TDigest at the same time as it's being updated by the record method. Since TDigest is not a thread safe object, this could result in either incorrect output from size, or possibly even unexpected exceptions |
I am not sure this is necessary. Tdigest is accessed through an atomicReference. Every operation is atomic so we will never missed any count. If locking is not an issue, we can synchonized every method and remove the atomicReference. I don't think we need sychronized and the atomic reference. |
One thread does this: Another thread does this: Having an |
That's correct. my bad. It has been a while :). I was more concern about thread safety around add / read operation. I was ok with size being off. I don't see a problem making getCount precise but that will increase context switching. What kind of stat are you getting on locking with this implementation ? |
I did not measure, and I am not sure how getCount is used - is it used for anything important? |
Almost every 30 seconds, whenever ffwdReporter reports metrics. |
Since TDigest objects are not thread safe, we need to protect reads from
getting corrupted data or potentially throwing exceptions.