Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add multiplexing data channel #43

Merged
merged 2 commits into from
May 24, 2023

Conversation

sjvanrossum
Copy link
Collaborator

This is a first pass at multiplexing data channels for source and sink operators.
I left a few TODOs to follow up on regarding:

  • Benchmarking DashMap against Moka or combining that with a simple hash map local to the multiplexer which retrieves from the concurrent map and stores the receiver based on (InstructionId, TransformId) pairs and evicting them when their last data/timers has been observed.
  • Using turmoil to simulate client and server environments in tests.

Additionally (not marked TODO), the mpsc channel is unbounded and that's fine for now since nothing's hooked up to it, so it's a little fuzzy to define a limit for a bounded channel.

Copy link
Owner

@laysakura laysakura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sjvanrossum Great work! Left only minor comments but LGTM overall.

sdks/rust/tests/data_test.rs Outdated Show resolved Hide resolved
sdks/rust/tests/data_test.rs Outdated Show resolved Hide resolved
sdks/rust/src/worker/mod.rs Outdated Show resolved Hide resolved
@laysakura laysakura merged commit 54b4d23 into laysakura:rust_sdk May 24, 2023
8 checks passed
@laysakura
Copy link
Owner

Thank you so much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants