Skip to content

Add support for algebraic stream joins#327

Open
albe wants to merge 2 commits into
mainfrom
new-storage-api
Open

Add support for algebraic stream joins#327
albe wants to merge 2 commits into
mainfrom
new-storage-api

Conversation

@albe

@albe albe commented Jul 3, 2026

Copy link
Copy Markdown
Owner

This pull request introduces a new, more expressive logical stream selector API for composing event streams, replacing the previous approach with a unified selector algebra. The core change is the adoption of nested array-based selectors that alternate between OR and AND logic by depth, enabling complex queries without a separate query language. The documentation and implementation have been updated to reflect this new model, including DCB (Domain Command Bus) query compatibility and guidance on using matchers versus tag streams.

Logical Stream Selector API and Implementation:

  • Introduced a selector algebra for fromStreams, where nested arrays alternate between OR and AND operators by depth, allowing for expressive composition of streams [1] [2] [3].
  • Removed the separate CombinedEventStream in favor of a single JoinEventStream implementation that evaluates selector trees using index set operations (union/intersect) [1] [2] [3] [4].

Documentation and API Updates:

  • Updated API documentation (docs/api.md, docs/streams.md, docs/architecture.md) to describe the new selector tree syntax, operator alternation, and examples for composing streams [1] [2] [3] [4].
  • Added changelog notes for version 1.4.0 to highlight the new selector algebra and DCB documentation updates.

DCB Query Compatibility and Best Practices:

  • Documented how DCB queries (using types and tags) are compiled into selector algebra, and clarified the implementation trade-offs between using tag streams and matcher-only approaches [1] [2] [3] [4] [5].
  • Provided guidance on when to use tag streams versus matcher logic, and described the migration and rollout plan for the new API [1] [2].

Internal and Validation Improvements:

  • Added a utility function assertSelectorStreamsExist to recursively validate that all streams referenced in a selector exist, improving error handling for nested selectors [1] [2].

Summary of Most Important Changes:

Logical Selector Algebra & Implementation:

  • Added selector algebra to fromStreams for nested OR/AND composition, deprecating CombinedEventStream in favor of a unified JoinEventStream implementation. [1] [2] [3]
  • Updated internal validation to ensure all streams in complex selectors exist using assertSelectorStreamsExist. [1] [2]

Documentation & API:

  • Revised API and architecture docs to explain the new selector model, including usage examples and operator alternation by depth. [1] [2] [3] [4]
  • Updated changelog for v1.4.0 to reflect these major changes.

DCB Query Support & Best Practices:

  • Documented DCB query translation to selector algebra, clarified matcher-only vs. tag stream approaches, and provided migration guidance. [1] [2] [3] [4] [5]

These changes provide a more powerful and flexible foundation for logical stream composition and DCB query compatibility.

albe added 2 commits June 28, 2026 00:39
It is now possible to join streams by selecting events occurring in any of, all of or a combination thereof, streams.
@coveralls

Copy link
Copy Markdown

Coverage Status

coverage: 98.257% (-0.06%) from 98.321% — new-storage-api into main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants