Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IAtomicLong corrupt value after memeber leaves cluster #6074

Closed
junder31 opened this issue Aug 27, 2015 · 3 comments

Comments

Projects
None yet
5 participants
@junder31
Copy link

commented Aug 27, 2015

It appears that under some circumstances Hazelcast is getting a corrupt value for IAtomicLongs when a member leaves the cluster.

Hazelcast version is 3.5.1 (I have also verified same behavior in 3.5 and 3.4)
Java version is 1.7u55

In our actual product we are attempting to use IAtomicLong to keep track of the number of concurrent file transfers to each destination we are sending to. Whenever one of the cluster members shut down we observe several of these counts becoming corrupt. The corrupted values always seem to be negative and generally are between -1 and -10000.

I have created a small test application that is able to produce the same issue.
https://github.com/junder31/IAtomicLongTest

In this application EdgeAnalog is an analog for the transfer destinations in our real app and FileTransferAnalog is an analog for the file transfers. In this test one thread pushes FileTransferAnalogs to a queue and multiple DistributionJobThreads consume the transfers. Each DistributionJobThread has its own hazlecastInstance. After 30 seconds of steady state I stop one of the DistributionJobThreads and observe several of the IAtomicLongs are corrupt on the next read.

In order to try to isolate the issue as a Hazelcast issue I also update a java.util.concurrent.atomic.AtomicLong in parallel to the Hazelcast IAtomicLong and log when they diverge.

@gurbuzali gurbuzali self-assigned this Aug 27, 2015

@mdogan mdogan added Type: Defect and removed PENDING labels Nov 2, 2015

@mdogan mdogan added this to the 3.6 milestone Nov 2, 2015

@gurbuzali gurbuzali modified the milestones: 3.7, 3.6 Nov 27, 2015

@gurbuzali gurbuzali removed their assignment Nov 30, 2015

@vbekiaris

This comment has been minimized.

Copy link
Contributor

commented May 24, 2016

This issue is no longer reproducible in 3.7-SNAPSHOT. Thanks @junder31 for providing the reproducer, it works correctly now.

@vbekiaris vbekiaris closed this May 24, 2016

@jerrinot

This comment has been minimized.

Copy link
Contributor

commented May 24, 2016

I reckon it was fixed by #7303

@vbekiaris

This comment has been minimized.

Copy link
Contributor

commented May 24, 2016

Well spotted, both 3.5.5 and 3.6 work fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.