periodic index-out-of-bounds error

[stealthrabbi]

Using v0.5.2

My program is able to run days sending binary pictures, but will occasionally encounter an IndexOutOfBoundsException. I am not sure if it's an apparent issue with the data being sent to the zmq client, or something with a connection. The issue is not easily repeatable.

Also, are there release notes for different versions? I see there's now a 0.5.3, but I can't tell what's been added to it.

Exception in thread "My Thread" Exception in thread "My Thread" java.lang.IndexOutOfBoundsException: Index 1 out of bounds for length 1
        at java.base/jdk.internal.util.Preconditions.outOfBounds(Unknown Source)
        at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Unknown Source)
        at java.base/jdk.internal.util.Preconditions.checkIndex(Unknown Source)
        at java.base/java.util.Objects.checkIndex(Unknown Source)
        at java.base/java.util.ArrayList.get(Unknown Source)
        at zmq.socket.pubsub.Dist.distribute(Dist.java:162)
        at zmq.socket.pubsub.Dist.sendToMatching(Dist.java:138)
        at zmq.socket.pubsub.XPub.xsend(XPub.java:253)
        at zmq.SocketBase.send(SocketBase.java:718)
        at org.zeromq.ZMQ$Socket.send(ZMQ.java:3205)
        at org.zeromq.ZFrame.send(ZFrame.java:136)
        at org.zeromq.ZFrame.sendAndKeep(ZFrame.java:151)
        at org.zeromq.ZMsg.send(ZMsg.java:187)
        at org.zeromq.ZMsg.send(ZMsg.java:159)
        at org.zeromq.proto.ZPicture.sendBinaryPicture(ZPicture.java:168)
        at org.zeromq.ZMQ$Socket.sendBinaryPicture(ZMQ.java:3342)
		<<my code>>

[trevorbernard]

@stealthrabbi Can you verify if this is still an issue with v0.5.3?

[stealthrabbi]

@trevorbernard I will try. The issue is that the exception sometimes takes many days to surface -- it's hard to pin down. Do you happen to have a list of things changed in 0.5.2 -> 0.5.3

[trevorbernard]

https://github.com/zeromq/jeromq/blob/master/CHANGELOG.md

[stealthrabbi]

No the issue still exists in 0.5.3. Took about 24 hours to recreate. In my program, I've added an exception handler for this case, but I see now that the exception continues forever. I am guessing I need to create a new zmq socket. I also don't know if I can reuse the ZContext I created when my program starts, or if a new ZContext is needed.

[fbacchella]

Just to be sure, the XPub socket is not shared between threads ?

[fbacchella]

Can you try with this branch: https://github.com/fbacchella/jeromq/tree/issue938 ?

[elihusmails]

I am still seeing this issue with version 0.5.4.

[fbacchella]

This branch was not merged as nobody provided any feebdback.

[elihusmails]

I added the changes to the Dist.java in my local 0.5.4 environment and recompiled/installed the change. While I'm not getting the stack trace, it appears that my app is silently failing now. I'm going to dig into this more.

[tommy2d]

I'm also experiencing this issue and for me it seems fairly easily reproduceable:

at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70) at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:266) at java.base/java.util.Objects.checkIndex(Objects.java:361) at java.base/java.util.ArrayList.get(ArrayList.java:427) at zmq.socket.pubsub.Dist.distribute(Dist.java:161) at zmq.socket.pubsub.Dist.sendToMatching(Dist.java:137) at zmq.socket.pubsub.XPub.xsend(XPub.java:253) at zmq.SocketBase.send(SocketBase.java:829) at zmq.SocketBase.send(SocketBase.java:792) at org.zeromq.ZMQ$Socket.send(ZMQ.java:3445)

In my setup, a Java application creates a zeromq socket of tpye SocketType.PUB and connects to a XPUB/XPUB router. The connection is setup in the constructor of a class, and the oob exception is triggered periodically (but i can see it happening at least once a day in my current setup).

[laichak]

I am also having this issue randomly with version 0.6.0

java.lang.IndexOutOfBoundsException: Index 2 out of bounds for length 2
at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:266)
at java.base/java.util.Objects.checkIndex(Objects.java:361)
at java.base/java.util.ArrayList.get(ArrayList.java:427)
at zmq.socket.pubsub.Dist.distribute(Dist.java:161)
at zmq.socket.pubsub.Dist.sendToMatching(Dist.java:137)
at zmq.socket.pubsub.XPub.xsend(XPub.java:253)
at zmq.SocketBase.send(SocketBase.java:829)
at zmq.SocketBase.send(SocketBase.java:792)
at org.zeromq.ZMQ$Socket.send(ZMQ.java:3445)
at org.zeromq.ZMQ$Socket.send(ZMQ.java:3408)

Looking at the source code in https://github.com/zeromq/jeromq/blob/master/jeromq-core/src/main/java/zmq/socket/pubsub/Dist.java

Around line 161 in Dist.java

//  Put the message to all active pipes.
private void distribute(Msg msg)
{
    //  If there are no matching pipes available, simply drop the message.
    if (matching == 0) {
        return;
    }

    // TODO isVsm

    //  Push copy of the message to each matching pipe.
    for (int idx = 0; idx < matching; ++idx) {
        if (!write(pipes.get(idx), msg)) {
            --idx; //  Retry last write because index will have been swapped
        }
    }
}

I think for some odd reason the int value of "matching" is larger than the size of the "pipes".

My suggestion is:

for (int idx = 0; idx < matching && idx < pipes.size(); ++idx) {

Thanks.