IMap listener unregistered after two failover/failbacks

[agherardi]

I have a 2-node cluster using Hazelcast opensource 3.8. My app uses an IMap and registers both an entry listener and a partition lost listener. The code is as follows:
String reg = map.addEntryListener(hzListener, true); String partLostReg = map.addPartitionLostListener(hzListener);

After 2 failovers (consisting of unplugging the Ethernet cable from one of the nodes) and 2 failbacks (consists of re-plugging the Ethernet cable), the listener no longer fires one the node that was disconnected/reconnected.

I took a heap dump on both nodes and noticed that, on the node that was connected/disconnected, my listener was garbage collected - Eclipse MAT shows a count of 0 instances of the listener's class. The listener still exists on the other node.

Any suggestions?

[agherardi]

Since it appears that topic and IMap listeners all end up registering with the event service, I'm wondering if this issue has the same root cause as #7742.

[agherardi]

I compared the heap dumps of the "bad" and "good" nodes. On both nodes, I see 3 com.hazelcast.spi.impl.eventserver.impl.Registration objects for my IMap, 2 have a MapPartitionLostEventFilter, one has an EventListenerFilter. However the value of Registration.listener is null on the "bad" node and non-null on the "good" node.

If I understand correctly, Registration.listener is null for registrations from a remote node. Indeed, on the "bad" node the subscriber has the IP of the "good" node; while on the "good" node, the subscriber is its own IP.

[agherardi]

This issue was caused by an application error.