COLA Tree indexes

[laa]

We successfully implemented sbtree index engine BUT it contains just a few optimizations for big data processing COLA trees are much faster in big data case (about 200 times faster on inserts in case of almost all content is places out or RAM and we have billions of records).
This issue requires to resolve several open questions:

1. Durability
2. Efficient multi core (it is even harder than multi threading) implementation.

My estimation is 3 months of full time development for single person and 2 months in pair but we will have a dramatic speedup on big data case.

This change have to be implemented after completion of plocal storage and multi core support for new disk cache.

[jankotek]

Hi, how is it going with COLA trees?

I work on MapDB and there is similar problem. Huge BTreeMap has very slow random updates. I am going to solve this by storing modification in second btree. Than merging two or more btrees using data pump. It is mostly streaming so should be reasonably fast.

But I would be very happy to provide testing and advice if someone wants to take COLA trees head on :-)

[CasperCL]

Any news on this? When is this going to be implemented?

[laa]

We need time to implement this. so now we work on smaller issues.

[tychota]

Any plan in near road or starmap ??

[laa]

Hi,

Now we concentrated on multicore scalability.
But then will work on mitigation of random IO overhead.

So for next several months we will not work on this issue.

On Thu, Sep 17, 2015 at 4:09 PM TychoTa notifications@github.com wrote:

Any plan in near road or starmap ??

—
Reply to this email directly or view it on GitHub
#1756 (comment)
.

[tychota]

Ty for answer ;)

[laa]

I think with new transaction system which we are working on problems listed in issue description will be solved automatically , so we will have all tools to implement given issue.

[saeedtabrizi]

@lvca , @laa is there any chance to have cola index in OrientDB 3 release ?

[laa]

@saeedtabrizi there are very big chances that we put it in 3.x release. I am working now on design to gather together all pieces of implementation of coal tree indexes. We will start with detailed design proposal during a couple of months and then will put it on our schedule.

[lvca]

This means it will be not available for 3.0, but later.

[saeedtabrizi]

@laa , @lvca it's a good news for orientdb lovers . 👍

[laa]

Not actual any more

[pontusvision]

Sorry, please explain what happened with this issue; have the changes been implemented, or are they no longer relevant? The documentation points to this page, but I cannot understand whether it has been implemented, or it has been cancelled.

[jankotek]

@pontusvision my understanding COLA trees (buffer trees) are obsolete by append only store such as RocksDB. I am not sure how OrientDB, but I never implemented it in MapDB.