Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is definitely a WIP (though I think it's relatively safe, as
these things go,) and I mostly wanted to see what it would take to do
something like this.
KDL is a pretty neat way of expressing "documents" and comes with a
pretty complete ecosystem of tools and the query language means this
is pretty easy to support without all that much heavy lifting.
I wanted to stop and get feedback before digging further in, and
obviously see what CI had to say for itself.
In my mind there were a couple of other interesting
things/possibilites that I wanted to mention, because I had been
thinking about it.
I really wanted to have a way of type alaising a field so that it
could have KDL data in it and GlareDB (and in particular the storage
engine) could think about optimizations within that. Adding UDFs
seems tractable, and while I think ultimately we'd want to store KDL
data in UTF-8 fields: it'd be nice to be able to error if someone
passed the wrong kind of field.
It's a project for another day, but it would be interesting to see
if you could make a binary format based on the data model:
length prefixes for more efficent skipping.
store the encoding format for numbers (int32 for number of 32bit
integers of storage; plus 32bits for a bitmask to store
[is_float;is_varint,is_string])
maybe use the sign bit of the length prefixes for "streaming" to
indicate if this length includes the end of
document/message/component or if after the provided length the
next length is a continuation of the same object.