com.orientechnologies.orient.core.index.OIndexException: Error during insertion of key in index

[satyasrc]

Expected behavior and actual behavior

OrientDB version used: 2.1.10
I am uploading full dbpedia dataset into orientDB. I have created a schema in which i have some properties defined on my custom vertex and edge classes. There are some Hash indices also defined on some of those properties. I am parsing the dbpedia graph from top to bottom in breadth-first fashion from Jena TDB and inserting into OrientDB. But i am getting this OIndexException after insertion of 27151 nodes and 30605 edges, though there was considerable amount of system-memory left. All indices used are of type HASH_INDEX (UNIQUE, NOT_UNIQUE, and FULL_TEXT). The full stack trace is given below:
Exception in thread "main" com.orientechnologies.orient.core.index.OIndexException: Error during insertion of key in index
at com.orientechnologies.orient.core.index.OIndexMultiValues.put(OIndexMultiValues.java:162)
at com.orientechnologies.orient.core.index.OIndexMultiValues.put(OIndexMultiValues.java:52)
at com.orientechnologies.orient.core.index.OIndexAbstract.putInSnapshot(OIndexAbstract.java:965)
at com.orientechnologies.orient.core.index.OIndexAbstract.applyIndexTxEntry(OIndexAbstract.java:1151)
at com.orientechnologies.orient.core.index.OIndexAbstract.addTxOperation(OIndexAbstract.java:762)
at com.orientechnologies.orient.core.tx.OTransactionOptimistic$CommitIndexesCallback.run(OTransactionOptimistic.java:101)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:1211)
at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:606)
at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:156)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2689)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2658)
at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.commit(ONetworkProtocolBinary.java:1225)
at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.executeRequest(ONetworkProtocolBinary.java:404)
at com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:223)
at com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:77)
Caused by: com.orientechnologies.orient.core.exception.OStorageException
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.put(OLocalHashTable.java:381)
at com.orientechnologies.orient.core.index.engine.OHashTableIndexEngine.put(OHashTableIndexEngine.java:153)
at com.orientechnologies.orient.core.index.OIndexMultiValues.put(OIndexMultiValues.java:155)
... 14 more
Caused by: java.lang.IllegalStateException: Extendible hashing tree in corrupted state.
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.getBucket(OLocalHashTable.java:1848)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1315)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413)
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.put(OLocalHashTable.java:373)
... 16 more

Running Mode

• Remote on the same machine.

OrientDB Version

• v2.1.10

Operating System

• Linux

Java Version

• 8

This is an urgent issue for us as we are trying to deploy OrientDB in our production environment.
Any help is much appreciated.
Thanks.

[laa]

Hi,
We have already fixed a similar issue in the latest version.
Could you try on 2.1.18 version and send us feedback whether issue fixed.

[satyasrc]

Ok, Let me try that.
Thanks.

[satyasrc]

I am trying the same with version 2.2.0. However, i am not able to see any of the already inserted data while insertion code is running. The graph is showing as empty while previously, when using v2.1.10 i was able to see the already inserted data even while the insertion code (the same code) was running on the db.
This is crucial for us, since this code would be running for days on the dbpedia & other linked datasets and we need to be able to see the already inserted data.
Any thoughts on why this is happening!! Is this a bug ? Remember, i am using remote mode for precisely the same reason, that is, so that i am able to see already inserted data while the graph upload is still happening.
Awaiting you reply.
Thanks.

[laa]

@srthecodexpress could you provide me code which you use ?

[satyasrc]

@laa unfortunately, I can't. The code uses our proprietary libraries. But i can tell you that insertion is happening as OrientDB is printing out the info on the console of each inserted node and edge. And, it is also printing out that the particular node/edge already exists when i am backtracking to insert next DBpedia subtree, so definitely it does exist in OrientDB. But i can't see it in OrientDB studio!! The DB is showing as empty there!! I should point out that i am using OrientGraphTx mode with AutoStartTx set to false, as i am manully starting and committing transactions before and after each node/edge, property insert.

[laa]

@maggiolo00 could you check that ?

[luigidellaquila]

Hi @srthecodexpress

Do you think you can share the DB (zipped db folder)? Could you also try to access the db from OrientDB Console (CLI) and see if you can query it?

Thanks

Luigi

[satyasrc]

Hi @laa @luigidellaquila
That problem is fixed. I had to set the setAutoStartTx property to false at the time of graph or factory creation itself whereas i was doing it after the graph creation in tx mode due to which a transaction was left open. This caused the other transaction commits to not show up in the db.

Thanks guys for your time and patience. 👍