Skip to content
This repository has been archived by the owner on Oct 30, 2024. It is now read-only.

Pull upstream changes #12

Merged
merged 8 commits into from
May 24, 2024
Merged

Conversation

antiguru
Copy link
Member

No description provided.

frankmcsherry and others added 8 commits May 8, 2024 13:35
This switches from consolidating each input individually to
consolidating at the output. The benefits are that it can yield better
consolidation performance because it can consolidate across input
containers instead of only being able to consolidate each individual
input container.

Signed-off-by: Moritz Hoffmann <antiguru@gmail.com>
Adds support for implementing semigroup for types that can plus_equals
from another type, for example vectors and slices.

Fixes a bug where the semigroup implementation for `Vec` would add
overhanging elements twice, i.e., `[1,2] + [1,1,1]` would result in
`[2,3,2`] instead of `[2,3,1]`.

This leaves one quirk where `is_zero` does not depend on `Rhs`, so Rust
cannot figure out which implementation to use when two are in scope,
forcing the caller to phrase it as `<R as Semigroup>::is_zero(&value)`.
This only applies if `R` implements `Semigroup + Semigroup<Other>`.

Signed-off-by: Moritz Hoffmann <antiguru@gmail.com>
Signed-off-by: Moritz Hoffmann <antiguru@gmail.com>
* Pass data from batcher to builder by chain

Currently, the data shared between the batcher and the builder are
individual tuples, either moved or by reference. This limits flexibility
around what kind of data can be provided to a builder, i.e., it has to
be in the form of tuples, either owned or a reference to a fully-formed
one. This works fine for vector-like structures, but will not work for
containers that like to arrange their data differently.

This change alters the contract between the batcher and the builder to
provide chunks instead of individual items (it does not require
_chains_.) The data in the chunks must be sorted, and subsequent calls
must maintain order, too. The input containers need to implement
`BuilderInput`, a type that describes how a container's items can be
broken into key, value, time, and diff, where key and value can be
references or owned data, as long as they can be pushed into the
underlying key and value containers.

The change has some quirks around comparing keys to keys already in the
builder. The types can differ, and the best solution I could come up
with was to add two explicit comparison functions to `BuilderInput` to
compare keys and values. While it is not elegant, it allows us to move
forward with this change, without adding nightmare-inducing trait bounds
all-over.

Signed-off-by: Moritz Hoffmann <antiguru@gmail.com>

* Address feedback

Signed-off-by: Moritz Hoffmann <antiguru@gmail.com>

---------

Signed-off-by: Moritz Hoffmann <antiguru@gmail.com>
* Introduce and integrate IntoOwned trait

* Respond to feedback
@antiguru antiguru merged commit 4ba7bc2 into MaterializeInc:master May 24, 2024
1 check failed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants