LINKLIST: provide a better serialized implementation for many items #1808

Open
lvca opened this Issue Nov 9, 2013 · 11 comments

Projects

None yet

4 participants

@lvca
OrientDB member

This is related to the issue #1807.

Right now the linklist is stored as [,]. This doesn't scale when you've many items because everytime OrientDB has to un-marshall the full list, add an item and re-marshall it. So we need something like we did for LINKSET but with LIST. The first idea in my mind is linked list, but probably there can be better impl.

@lvca
OrientDB member

Can we use LINKBAG for this?

@laa
OrientDB member
@lvca
OrientDB member

On this case the index would be the position, not a document field. Could it be efficient?

@laa
OrientDB member
@laa
OrientDB member
@enisher

The same issue with insert into position other then the last.
I'm not sure that such batch updating of indexing will provide a good scale.

There is a workaround, we could use floats as keys and don't update other indexes. So the key will not represent an index just some value to order elements.
However it is rather workaround then proper solution.

WDYT about creation a new disk data structure, like SBTreeBonsai?

@laa
OrientDB member
@tglman
OrientDB member

is it done anything on this ? this is closely related to #681, the new serialization format should support partial update

@lvca
OrientDB member

My initial idea was to ave huge lists split in multiple records as linked list or whatever.

@lvca
OrientDB member

@tglman WDYT?

@lvca lvca modified the milestone: 2.0rc1, 2.0rc2 Aug 28, 2014
@tglman tglman was assigned by lvca Aug 28, 2014
@tglman tglman modified the milestone: 2.1, 2.0-M2 Sep 22, 2014
@tglman
OrientDB member

the implementation in the record binary serialization, already improve the LINKLIST handling compared to the csv serialization. in future we may consider to user the ridbag for really uge list.

moving forward!

@lvca lvca modified the milestone: 2.2, 2.1 Feb 1, 2015
@lvca lvca modified the milestone: 3.0, 2.2 Apr 14, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment