Operator for join

[tantaman]

Join is a bit verbose to test. 900 lines of tests 😅.

Next PR will expose JOIN to ZQL proper.

Left and Right join can be done without much extra work (add a row to the result set, rather than skipping, if a match is not present in the left/right table).

Commits shouldn't be reviewed in isolation.

---

Kept for history:

Decided on a nest operator that can appear in the position of a selected column as well as supporting normal joins.

Fiddling with the best way to alias tables until then. E.g.,

Issue.as('t1')
  .select('t1.title')
  .join(Issue, 'parent_id', 'id').as('t2')
  .where('t2.status', '=', 'blocked')

vs

Issue.select('title').join(Issue, 't2', 'parent_id', 'id').where('t2.status', '=', 'blocked')

Or do all joins as nested joins 🤔 and the where / select parts becomes a path

Issue.select('title', 'parent.title')
  .nest(Issue, 'parent', 'parent_id', 'id')
  .where('issue.parent.status', '=', 'blocked');

Alternate nest join ideas:

Issue.select(
  'title',
  nest('parent', Issue.select(...)).on('parent_id', 'id')
).where('issue.parent.status', '=', 'blocked');

Issue.select(
  'title',
   nest('owner', User.select('name'))
     .on('owner_id', 'id')
)

I think the nest idea is cleanest.

1. It can replace correlated sub-queries for most use cases
2. Where clauses become simpler since the user can operate on the entire aggregate

Drawbacks:

1. Copying the outer object to nest something inside
2. The nested collection is copied if something inside of it changes
3. Junction tables show up in the results

[arv]

How does this happen again?

[tantaman]

for operators with more than one input. We send the commit notification down the graph so it'll get a commit notification twice.

An alternative would be to not send commit down the graph and instead have every effect and view register with the global materialite instance if they're run in a given transaction. On commit, they'd get notified directly.

[tantaman]

Working on a separate PR that does not send commit through the graph. Only effects and views need to know of commits so they can call their listeners so it makes sense not to send commit through the graph from a saving work perspective.

[arv]

remove

[arv]

please add some tests where newDifference is called more than once per stream and version.

[tantaman]

good catch. I need to take version into account in the operator to handle this case.

[tantaman]

This was actually working fine. I wrote the test case incorrectly 😅. New test added for this case that actually does the correct thing.

[arv]

LGTM

[tantaman]

updated to always iterate over the smallest collection in the outer loop.

This caused our synthetic ids for join rows to be non-deterministic (outer loop row was concatenated first then inner loop row) so add a sort step to ensure joins always produce the same id for the same two rows.

[arv]

It is generally better to use module level functions than private methods. Only use private methods when you need access to other (private) state.

[arv]

Still LGTM