Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Using YoctoDB database consists of the following distinct phases:
- Add documents to a database in append-only mode
- Build and serialize database
- Open serialized database (or composite database) in read-only mode
Let's consider general YoctoDB notions.
YoctoDB document is an opaque user provided sequence of bytes augmented with indexing fields used for filtering and/or sorting on the last phase.
We use Google Protocol Buffers to serialize business entities to sequence of bytes, but it is not obligatory.
Index field value is a named comparable byte sequence attached to the document before adding the document to a database being constructed.
There are several types of index fields:
FILTERABLE-- the field can be used to filter documents
SORTABLE-- the field can be used to sort documents
Different filterable field values with the same name can be attached to the same document to allow filtering based on multivalued fields.
Database serialization involves building efficient read-only search structures based on all the accumulated information (primarily index field values).
Opening a serialized database in read-only mode costs nothing except possible disk cache warm up and checksum test if necessary.
YoctoDB query language supports the following features:
- Filtering documents using several fields (
in, range queries)
- Condition combining using logical operators (
- Sorting documents using several fields (
- Skip and limit
- Filtered sorted document payload processing using user provided callback
YoctoDB query execution engine is based on
Current implementation uses thread local cached
BitSet pools for each database.
Memory consumption can be estimated using the following formula:
thread_count * max(3, query_depth) * document_count / 64
There is no query optimizer so a query is executed as is.
YoctoDB composite database provides the same database search features considered above for a set of basic databases.
If you decide to build a partitioned database you should base your partitioning criteria on future request patterns so that query execution engine could skip entire partitions.
See Getting Started Guide to get code examples.