Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(kdl): kdl_match function #2145

Merged
merged 6 commits into from
Dec 4, 2023
Merged

feat(kdl): kdl_match function #2145

merged 6 commits into from
Dec 4, 2023

Conversation

tychoish
Copy link
Collaborator

This is definitely a WIP (though I think it's relatively safe, as
these things go,) and I mostly wanted to see what it would take to do
something like this.

KDL is a pretty neat way of expressing "documents" and comes with a
pretty complete ecosystem of tools and the query language means this
is pretty easy to support without all that much heavy lifting.

I wanted to stop and get feedback before digging further in, and
obviously see what CI had to say for itself.

In my mind there were a couple of other interesting
things/possibilites that I wanted to mention, because I had been
thinking about it.

  • I really wanted to have a way of type alaising a field so that it
    could have KDL data in it and GlareDB (and in particular the storage
    engine) could think about optimizations within that. Adding UDFs
    seems tractable, and while I think ultimately we'd want to store KDL
    data in UTF-8 fields: it'd be nice to be able to error if someone
    passed the wrong kind of field.

  • It's a project for another day, but it would be interesting to see
    if you could make a binary format based on the data model:

    • length prefixes for more efficent skipping.

    • store the encoding format for numbers (int32 for number of 32bit
      integers of storage; plus 32bits for a bitmask to store
      [is_float;is_varint,is_string])

    • maybe use the sign bit of the length prefixes for "streaming" to
      indicate if this length includes the end of
      document/message/component or if after the provided length the
      next length is a continuation of the same object.

@tychoish tychoish enabled auto-merge (squash) December 4, 2023 20:59
@tychoish tychoish merged commit 6cfa754 into main Dec 4, 2023
10 checks passed
@tychoish tychoish deleted the tycho/kdl-objects branch December 4, 2023 21:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants