Chunk ... no longer exists

[ptoews]

I'm using Nitrite database to store a million documents and then perform different read operations on them (for benchmarking).
After storing the documents, while trying to read all documents via collection.find().iterator(), I get the following exception:

java.lang.IllegalStateException: Chunk 162 no longer exists [1.4.196/9]
	at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:765)
	at org.h2.mvstore.MVStore.getChunkIfFound(MVStore.java:953)
	at org.h2.mvstore.MVStore.getChunk(MVStore.java:935)
	at org.h2.mvstore.MVStore.readPage(MVStore.java:1943)
	at org.h2.mvstore.MVMap.readPage(MVMap.java:741)
	at org.h2.mvstore.Page.getChildPage(Page.java:217)
	at org.h2.mvstore.Cursor.fetchNext(Cursor.java:150)
	at org.h2.mvstore.Cursor.next(Cursor.java:50)
	at org.dizitart.no2.internals.DocumentCursor$DocumentCursorIterator.nextMatch(DocumentCursor.java:115)
	at org.dizitart.no2.internals.DocumentIterator.next(DocumentIterator.java:54)
	at org.dizitart.no2.internals.DocumentIterator.next(DocumentIterator.java:32)
        ...

Then, the next smaller read operations return more values than they should.(Not related to this problem)
I create the database with the default options and three indexes, and it occurs with and without enabled compressing. The used version is 2.1.0.

Is a million documents too much for Nitrite? The same data and test cases work fine with pure MVStore. Inserting only ~340 entries works fine too.

[anidotnet]

Could you please post the complete code here for both nitrite and mvstore alone to reproduce this error?

[ptoews]

I've tested it with another dataset with ~270k entries that worked fine as well.
The project is really big and many classes are involved, so I hope the adapter classes are enough for you:

NitriteDB:
Constructing DB:

Nitrite db = Nitrite.builder().compressed()
			.filePath(storageDir + File.separatorChar + DB_FILE).openOrCreate();
NitriteCollection collection = db.getCollection(COLLECTION_NAME);
this.collectionAccessor = new CollectionAccessor(db, collection);

CollectionAccessor.java: https://pastebin.com/N63681vg
IteratorWrapper.java: https://pastebin.com/vpeXUMcE

MVStore:
H2MVStoreMultiMap.java: https://pastebin.com/V8wsUXP2
IteratorWrapper is basically the same, without the document converting.

[anidotnet]

MVStore example is not giving this exception is because you are creating only one map there. But in nitrite, it creates 6 maps - 1 for collection, 3 for indices, 2 for meta information.

Can you please do me a favor and create 4 mvmaps in mvstore example and insert data in all of them simultaneously in the same fashion? I just need to confirm if it is the volume of data that is creating problem or not. I might take this case to mvstore developers, so it will be better if I give as much info as possible.

[ptoews]

I forgot to mention that H2MVStoreMutliMap class is instantiated three times.
But why do you want me to create only 4 mvmaps? What are the other two for then, because indexing should be enabled there too.

However, instantiating my MVStore multimap both 4 and 6 times works without problems.

[anidotnet]

I just want to make sure, you are creating 4 mvmaps in the same store and inserting 1 million of records in each of them parallely right?

I asked for 4 mvmaps, as other 2 are meta maps, it contains only few records, so does not contribute to this problem at all.

[ptoews]

I just want to make sure, you are creating 4 mvmaps in the same store and inserting 1 million of records in each of them parallely right?

Yes, exactly.

[anidotnet]

Thank you. Let me re-create the scenario and see what can be done about this.

[ptoews]

I'm currently trying to reduce the code for reproducing the error as well, and it looks like it depends on an outer layer that I didn't include here. I'll keep you updated if there maybe is a mistake on my side.

[ptoews]

Nevermind that, I just didn't get the exception sometimes randomly.
So here is the reduced code that definitely threw one for me:
https://pastebin.com/kk9P06sG

[anidotnet]

Thanks a lot for the time. I am looking into it.

[anidotnet]

Fix is ready in 2.1.1-SNAPSHOT. Please test and confirm

[ptoews]

I'm using Nitrite via Maven, but it's not available there yet?

[anidotnet]

https://oss.sonatype.org/content/repositories/snapshots/org/dizitart/nitrite/2.1.1-SNAPSHOT/

You need to add snapshot repo https://oss.sonatype.org/content/repositories/snapshots in maven pom

[ptoews]

Thanks. Yes, now it works.