DB file format changes

[tzaeschke]

This is a compound issue to collect planned/required file format changes.

Root Pages:

• Move commit counter into protected block
• Page version info
• root page counter (leave at 2 for now)

Index management:

• Specify Index class in stored index to support different index implementations (and versions)
• Support for different String-magic-number algorithms (implement via different index class?)
• Support for multi-field indexes
• Support for aggregate indexes across objects (path index) or classes (really?)

Index capabilities:

• Use prefix sharing index as standard
• Store min/max/cnt values for each subtree -> provide aggregate methods (min/max/avg/cnt/...)

Page storage:

• Store page offset of objects in page and store only page-id in OID index
• Page overflow: stop splitting int/long/double/...! We should only split arrays/string. This avoids allocating ByteBuffers and arrays.
• Dow we really need to align multi-byte entries to their multiples? May be useful for arrays (faster access). But for primitives?

Object storage:

• Store characters/string with 2 bytes per char (currently one byte ASCII). Also, make this configurable. Actually, Java 9 automatically stores Strings internally as 1-byte, unless 2-bytes are required. Can we do the same?
• Allow storing enums as String i.o. 'int'
• Do not store Schema-OID of target object
• Fix NULL storage, see Data(De)Serializer
• Replace standard collections with ZooDB collections: Avoid materializing objects when materializing a collection
• Collection storage format: Allow queries (map.get(x) == y) etc on serverside without materializing the collection. Store collections such that this is possible.

Schema:

• Support for 'named' fields (via @column etc).
• Support for @Embedded, may require reference to SCO/Embedded class
• Support for initialization value for schema evolution
• Support for initialization functions (at least preliminary, provide simple Object-ref?)
• Support for Field storage options, such as String (ASCII, UTF-8, UTF-16, ...), enum (int, String, ...), ...