Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add synchronized around getCount dist usage #114

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

spkrka
Copy link
Member

@spkrka spkrka commented Jul 16, 2021

Since TDigest objects are not thread safe, we need to protect reads from
getting corrupted data or potentially throwing exceptions.

Since TDigest objects are not thread safe, we need to protect reads from
getting corrupted data or potentially throwing exceptions.
Copy link

@storozhukBM storozhukBM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@ao2017 ao2017 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I may have lost track of thing. Why do we need synchronized around distRef ? It is an atomic object .

@spkrka
Copy link
Member Author

spkrka commented Jul 16, 2021

Without this change, it is possible for one thread to read the size of the TDigest at the same time as it's being updated by the record method. Since TDigest is not a thread safe object, this could result in either incorrect output from size, or possibly even unexpected exceptions

@ao2017
Copy link
Contributor

ao2017 commented Jul 16, 2021

Without this change, it is possible for one thread to read the size of the TDigest at the same time as it's being updated by the record method. Since TDigest is not a thread safe object, this could result in either incorrect output from size, or possibly even unexpected exceptions

I am not sure this is necessary. Tdigest is accessed through an atomicReference. Every operation is atomic so we will never missed any count.

If locking is not an issue, we can synchonized every method and remove the atomicReference. I don't think we need sychronized and the atomic reference.

@spkrka
Copy link
Member Author

spkrka commented Jul 16, 2021

One thread does this:
distRef.get().add(..)

Another thread does this:
distRef.get().size()

Having an AtomicReference does not prevent add(..) from being called at the same time as size()

@ao2017
Copy link
Contributor

ao2017 commented Jul 16, 2021

One thread does this:
distRef.get().add(..)

Another thread does this:
distRef.get().size()

Having an AtomicReference does not prevent add(..) from being called at the same time as size()

One thread does this:
distRef.get().add(..)

Another thread does this:
distRef.get().size()

Having an AtomicReference does not prevent add(..) from being called at the same time as size()

That's correct. my bad. It has been a while :). I was more concern about thread safety around add / read operation. I was ok with size being off. I don't see a problem making getCount precise but that will increase context switching. What kind of stat are you getting on locking with this implementation ?

@spkrka
Copy link
Member Author

spkrka commented Jul 16, 2021

I did not measure, and I am not sure how getCount is used - is it used for anything important?

@ao2017
Copy link
Contributor

ao2017 commented Jul 16, 2021

I did not measure, and I am not sure how getCount is used - is it used for anything important?

Almost every 30 seconds, whenever ffwdReporter reports metrics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants