New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why my arangodb exit when i use async java driver to do CRUD test? I use docker 3.1.7 image #2245

Closed
brucejee opened this Issue Jan 3, 2017 · 3 comments

Comments

Projects
None yet
3 participants
@brucejee

brucejee commented Jan 3, 2017

my environment running ArangoDB

I'm using the latest ArangoDB of the respective release series:

  • 2.8
  • 3.0
  • 3.1.7

On this operating system:

  • DCOS on
    • AWS
    • Azure
    • own infrastructure
  • Linux
    • Debian .deb
    • Ubuntu .deb
    • SUSE .rpm
    • RedHat .rpm
    • Fedora .rpm
    • Gentoo
    • docker - official docker library
    • other:
  • Windows, version:
  • MacOS, version:

I'm issuing AQL via:

  • web interface with this browser: running on this OS:
  • arangosh
  • arangodb-java-driver-async

My test enviroment is only one instance of docker 3.1.7 image. There is no error log in server's log.

image

Server log
image

Client log

INIT succNum:1, failNum:0, exceptNum:0
Insert testing...
INSERT qps:37330.844, consumeTime:2143ms, succTotal:80000, failTotal:0, exceptTotal:0
Update testing...
UPDATE qps:34349.508, consumeTime:2329ms, succTotal:80000, failTotal:0, exceptTotal:0
Select testing...
2017-01-03 19:31:12,253 ERROR [com.arangodb.internal.CollectionCache] - java.io.IOException: Reached the end of the stream.
com.arangodb.ArangoDBException: java.io.IOException: Reached the end of the stream.
at com.arangodb.internal.velocystream.ConnectionSync.write(ConnectionSync.java:111)
at com.arangodb.internal.velocystream.CommunicationSync.send(CommunicationSync.java:137)
at com.arangodb.internal.velocystream.CommunicationSync.execute(CommunicationSync.java:119)
at com.arangodb.internal.velocystream.CommunicationSync.execute(CommunicationSync.java:42)
at com.arangodb.internal.ArangoExecutorSync.execute(ArangoExecutorSync.java:58)
at com.arangodb.internal.ArangoExecutorSync.execute(ArangoExecutorSync.java:47)
at com.arangodb.ArangoCollection.getInfo(ArangoCollection.java:642)
at com.arangodb.internal.CollectionCache.execute(CollectionCache.java:100)
at com.arangodb.internal.CollectionCache.getInfo(CollectionCache.java:89)
at com.arangodb.internal.CollectionCache.getCollectionName(CollectionCache.java:76)
at com.arangodb.internal.velocypack.VPackConfigure$2.deserialize(VPackConfigure.java:84)
at com.arangodb.internal.velocypack.VPackConfigure$2.deserialize(VPackConfigure.java:74)
at com.arangodb.velocypack.VPack.getValue(VPack.java:352)
at com.arangodb.velocypack.VPack.deserializeObject(VPack.java:271)
at com.arangodb.velocypack.VPack.getValue(VPack.java:373)
at com.arangodb.velocypack.VPack.deserializeMap(VPack.java:422)
at com.arangodb.velocypack.VPack.getValue(VPack.java:367)
at com.arangodb.velocypack.VPack.deserialize(VPack.java:241)
at com.arangodb.velocypack.VPack$2.deserialize(VPack.java:219)
at com.arangodb.internal.velocypack.VPackDeserializers$4.deserialize(VPackDeserializers.java:92)
at com.arangodb.internal.velocypack.VPackDeserializers$4.deserialize(VPackDeserializers.java:86)
at com.arangodb.velocypack.VPack.getValue(VPack.java:352)
at com.arangodb.velocypack.VPack.deserialize(VPack.java:241)
at com.arangodb.util.ArangoUtil.deserialize(ArangoUtil.java:67)
at com.arangodb.internal.ArangoExecutor.deserialize(ArangoExecutor.java:112)
at com.arangodb.internal.ArangoExecutor.createResult(ArangoExecutor.java:108)
at com.arangodb.internal.ArangoExecutorAsync.lambda$execute$0(ArangoExecutorAsync.java:48)
at com.arangodb.internal.ArangoExecutorAsync.lambda$execute$1(ArangoExecutorAsync.java:56)
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at com.arangodb.internal.velocystream.CommunicationAsync.lambda$execute$0(CommunicationAsync.java:143)
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at com.arangodb.internal.velocystream.MessageStore.consume(MessageStore.java:56)
at com.arangodb.internal.velocystream.ChunkStore.checkCompleteness(ChunkStore.java:66)
at com.arangodb.internal.velocystream.ChunkStore.checkCompleteness(ChunkStore.java:60)
at com.arangodb.internal.velocystream.ConnectionAsync.lambda$open$0(ConnectionAsync.java:113)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Reached the end of the stream.
at com.arangodb.internal.velocystream.Connection.readBytesIntoBuffer(Connection.java:212)
at com.arangodb.internal.velocystream.Connection.readBytes(Connection.java:204)
at com.arangodb.internal.velocystream.Connection.readChunk(Connection.java:181)
at com.arangodb.internal.velocystream.ConnectionSync.write(ConnectionSync.java:98)
... 45 more
2017-01-03 19:31:12,341 ERROR [com.arangodb.internal.velocystream.MessageStore] - Connection reset
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:209)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at com.arangodb.internal.velocystream.Connection.readBytesIntoBuffer(Connection.java:210)
at com.arangodb.internal.velocystream.ConnectionAsync.lambda$open$0(ConnectionAsync.java:111)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2017-01-03 19:31:12,341 ERROR [com.arangodb.internal.velocystream.MessageStore] - Connection reset
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:209)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at com.arangodb.internal.velocystream.Connection.readBytesIntoBuffer(Connection.java:210)
at com.arangodb.internal.velocystream.ConnectionAsync.lambda$open$0(ConnectionAsync.java:111)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2017-01-03 19:31:12,340 ERROR [com.arangodb.internal.velocystream.MessageStore] - Reached the end of the stream.
java.io.IOException: Reached the end of the stream.
at com.arangodb.internal.velocystream.Connection.readBytesIntoBuffer(Connection.java:212)
at com.arangodb.internal.velocystream.ConnectionAsync.lambda$open$0(ConnectionAsync.java:111)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2017-01-03 19:31:12,253 ERROR [com.arangodb.internal.velocystream.MessageStore] - Reached the end of the stream.
java.io.IOException: Reached the end of the stream.
at com.arangodb.internal.velocystream.Connection.readBytesIntoBuffer(Connection.java:212)
at com.arangodb.internal.velocystream.ConnectionAsync.lambda$open$0(ConnectionAsync.java:111)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2017-01-03 19:31:12,345 ERROR [com.arangodb.internal.velocystream.MessageStore] - Connection reset
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:209)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at com.arangodb.internal.velocystream.Connection.readBytesIntoBuffer(Connection.java:210)
at com.arangodb.internal.velocystream.ConnectionAsync.lambda$open$0(ConnectionAsync.java:111)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2017-01-03 19:31:12,346 ERROR [com.arangodb.internal.velocystream.MessageStore] - Connection reset
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:209)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at com.arangodb.internal.velocystream.Connection.readBytesIntoBuffer(Connection.java:210)
at com.arangodb.internal.velocystream.ConnectionAsync.lambda$open$0(ConnectionAsync.java:111)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

My test code is very simple

private TestTaskResult initArangoDB() {
    TestTaskResult result = new TestTaskResult();
    try {
        String testDBName = configManager.getString(ConfigKey.TEST_DB_NAME);
        database = arangoDB.db(testDBName);
        result.addSuccessNum(1);
    } catch (ArangoDBException e) {
        e.printStackTrace();
        result.addExceptNum(1);
        result.addFailNum(1);
    }
    return result;
 }

private TestTaskResult testArangoInsert() {
    TestTaskResult result = new TestTaskResult();
    ArrayList<CompletableFuture<DocumentCreateEntity<BaseDocument>>> list = new ArrayList(threadTestCnt);
    ArangoCollectionAsync entityCollection = database.collection(ENTITY_COLLECTION);

    try {
        String nanoTime = String.valueOf(System.nanoTime());
        StringBuilder randomValue = createRandomCharData(propertyBytes);

        for (int i = 0; i < threadTestCnt; i++) {
            final BaseDocument doc = new BaseDocument();
            doc.setKey(namePrefix + String.valueOf(i + 1 + nameIdOffset));
            for (int j = 0; j < propertyNum; j++) {
                String appendProp = "__" + nanoTime + "__" + String.valueOf(i + 1) + "__" + String.valueOf(j + 1);
                randomValue.append(appendProp);
                doc.addAttribute("property" + String.valueOf(j), randomValue.toString());
                int len = randomValue.length();
                randomValue.delete(len - appendProp.length(), len);
            }

            final CompletableFuture<DocumentCreateEntity<BaseDocument>> f = entityCollection.insertDocument(doc);
            list.add(f);
        }
    } catch (ArangoDBException e) {
        e.printStackTrace();
    }

    for (int i = 0; i < threadTestCnt; i++) {
        try {
            DocumentCreateEntity<BaseDocument> createEntity = list.get(i).get();
            if (createEntity.getKey() != null) {
                result.addSuccessNum(1);
            } else {
                result.addFailNum(1);
            }
        } catch (InterruptedException | ExecutionException e) {
            result.addFailNum(1);
        }
    }

    return result;
}

private TestTaskResult testArangoUpdate() {
    TestTaskResult result = new TestTaskResult();
    int updatePropNum = propertyNum;
    if (propertyNum >= 10) {
        updatePropNum = propertyNum / 3;
    }

    ArrayList<CompletableFuture<DocumentUpdateEntity<BaseDocument>>> list = new ArrayList(threadTestCnt);
    ArangoCollectionAsync entityCollection = database.collection(ENTITY_COLLECTION);
    try {
        String nanoTime = String.valueOf(System.nanoTime());
        StringBuilder randomValue = createRandomCharData(propertyBytes);
        randomValue.append("_Update");

        for (int i = 0; i < threadTestCnt; i++) {
            final BaseDocument doc = new BaseDocument();
            doc.setKey(namePrefix + String.valueOf(i + 1 + nameIdOffset));

            for (int j = 0; j < updatePropNum; j++) {
                String appendProp = "__" + nanoTime + "__" + String.valueOf(i + 1) + "__" + String.valueOf(j + 1);
                randomValue.append(appendProp);
                doc.addAttribute("property" + String.valueOf(j), randomValue.toString());
                int len = randomValue.length();
                randomValue.delete(len - appendProp.length(), len);
            }

            final CompletableFuture<DocumentUpdateEntity<BaseDocument>> f = entityCollection.updateDocument(doc.getKey(), doc);
            list.add(f);
        }
    } catch (ArangoDBException e) {
        e.printStackTrace();
    }

    for (int i = 0; i < threadTestCnt; i++) {
        try {
            DocumentUpdateEntity<BaseDocument> updateEntity = list.get(i).get();
            if (updateEntity.getKey() != null) {
                result.addSuccessNum(1);
            } else {
                result.addFailNum(1);
            }
        } catch (InterruptedException | ExecutionException e) {
            result.addFailNum(1);
        }
    }

    return result;
}

private TestTaskResult testArangoSelect() {
    TestTaskResult result = new TestTaskResult();
    ArrayList<CompletableFuture<BaseDocument>> list = new ArrayList(threadTestCnt);
    ArangoCollectionAsync entityCollection = database.collection(ENTITY_COLLECTION);
    try {
        for (int i = 0; i < threadTestCnt; i++) {
            final CompletableFuture<BaseDocument> f = entityCollection.getDocument(namePrefix + String.valueOf(i + 1 + nameIdOffset), BaseDocument.class);
            list.add(f);
        }
    } catch (ArangoDBException e) {
        e.printStackTrace();
    }

    for (int i = 0; i < threadTestCnt; i++) {
        try {
            BaseDocument document = list.get(i).get();
            if (document.getKey() != null) {
                result.addSuccessNum(1);
            } else {
                result.addFailNum(1);
            }
        } catch (InterruptedException | ExecutionException e) {
            result.addFailNum(1);
        }
    }

    return result;
}

private TestTaskResult testArangoDelete() {
    TestTaskResult result = new TestTaskResult();
    ArrayList<CompletableFuture<DocumentDeleteEntity<Void>>> list = new ArrayList(threadTestCnt);
    ArangoCollectionAsync entityCollection = database.collection(ENTITY_COLLECTION);
    try {
        for (int i = 0; i < threadTestCnt; i++) {
            final CompletableFuture<DocumentDeleteEntity<Void>> f = entityCollection.deleteDocument(namePrefix + String.valueOf(i + 1 + nameIdOffset));
            list.add(f);
        }
    } catch (ArangoDBException e) {
        e.printStackTrace();
    }

    for (int i = 0; i < threadTestCnt; i++) {
        try {
            DocumentDeleteEntity<Void> deleteEntity = list.get(i).get();
            if (deleteEntity.getKey() != null) {
                result.addSuccessNum(1);
            } else {
                result.addFailNum(1);
            }
        } catch (InterruptedException | ExecutionException e) {
            result.addFailNum(1);
        }
    }

    return result;
}

@brucejee brucejee changed the title from Why my arangodb exit when i use async java driver to do CRUD test? to Why my arangodb exit when i use async java driver to do CRUD test? I use docker 3.1.7 image Jan 3, 2017

@mpv1989

This comment has been minimized.

Show comment
Hide comment
@mpv1989

mpv1989 Jan 12, 2017

Contributor

Hi,
the error suggests that the arango instance/docker-container is crashed.

Your code looks good and worked for me without any problems. I will try to reproduce it.

Contributor

mpv1989 commented Jan 12, 2017

Hi,
the error suggests that the arango instance/docker-container is crashed.

Your code looks good and worked for me without any problems. I will try to reproduce it.

@brucejee

This comment has been minimized.

Show comment
Hide comment
@brucejee

brucejee Jan 13, 2017

Thanks, I need to see if there are any coredump files. I remember it's ok when do small amount operations for every thread, but arangodb exit when client do large amount operations for every thread, probably 5000 CRUDs for every thread.

brucejee commented Jan 13, 2017

Thanks, I need to see if there are any coredump files. I remember it's ok when do small amount operations for every thread, but arangodb exit when client do large amount operations for every thread, probably 5000 CRUDs for every thread.

@dothebart

This comment has been minimized.

Show comment
Hide comment
@dothebart

dothebart Jan 26, 2017

Contributor

Hi, we did some testing with your sample code, and found an issue which was fixed with the now released ArangoDB 3.1.9; However, we did not re-test it dockerized.
If you can revalidate that its fixed for you too, I'd like to close this bug.

Contributor

dothebart commented Jan 26, 2017

Hi, we did some testing with your sample code, and found an issue which was fixed with the now released ArangoDB 3.1.9; However, we did not re-test it dockerized.
If you can revalidate that its fixed for you too, I'd like to close this bug.

@dothebart dothebart closed this Feb 21, 2017

@dothebart dothebart added this to the 3.1.9 milestone Feb 21, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment