fix(connectors): hand off SDK worker during blocking send callbacks - #3797
Open
mlevkov wants to merge 1 commit into
Open
fix(connectors): hand off SDK worker during blocking send callbacks#3797mlevkov wants to merge 1 commit into
mlevkov wants to merge 1 commit into
Conversation
|
Thanks for the PR. It is labeled Slash commands (own line, regular comment) move it around the queue:
See CONTRIBUTING.md for details. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #3797 +/- ##
=============================================
- Coverage 75.72% 51.07% -24.66%
Complexity 969 969
=============================================
Files 1322 1321 -1
Lines 159363 139283 -20080
Branches 132746 112666 -20080
=============================================
- Hits 120684 71136 -49548
- Misses 35041 65177 +30136
+ Partials 3638 2970 -668
🚀 New features to boost your workflow:
|
|
|
Contributor
Author
|
/request-review @hubcio |
The send callback runs synchronously inside the SDK's polling task, on a tokio runtime shared by every connector instance loaded from the same plugin library. A callback that blocks for backpressure pins one worker for the duration, and with enough saturated instances iggy_source_close for a sibling waits behind them, since the close blocks on the sibling's polling task getting scheduled to observe its shutdown signal. Wrap the callback in tokio::task::block_in_place so the worker is handed off before the callback runs. The SDK runtime is multi-threaded, which block_in_place requires. No FFI or ABI change; plugins pick this up when rebuilt against the updated SDK. Fixes apache#3796. Co-authored-by: Claude <noreply@anthropic.com>
mlevkov
force-pushed
the
sdk-callback-worker-handoff
branch
from
August 2, 2026 21:40
c73a8f5 to
2772550
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #3796. The send callback runs synchronously inside the SDK's polling task (
handle_messagesincore/connectors/sdk/src/source.rs), on the tokio runtime shared by every connector instance loaded from the same plugin library (static RUNTIME: OnceLock<Runtime>inlib.rs). A callback that blocks pins one worker for the duration. With #3795's bounded forwarding channel, a callback legitimately blocks for backpressure while the channel is full, so enough saturated instances of one library can occupy all workers, andiggy_source_closefor a sibling instance then waits behind them: the close blocks on the sibling's polling task getting scheduled to observe its shutdown signal.Change
One call site: wrap the callback invocation in
tokio::task::block_in_place, so the worker is handed off before the callback runs and the runtime keeps scheduling sibling tasks regardless of how long the callback blocks.Notes from the analysis in #3796:
block_in_placeconsults the calling thread's tokio context, which is only set on the plugin runtime's own worker threads.Runtime::new(), i.e. multi-threaded, whichblock_in_placerequires.This composes with #3795 (bounded channel + shutdown signaling) but does not depend on it: any long-blocking callback benefits.
Test plan
cargo clippy -p iggy_connector_sdk --all-targets --all-features -- -D warningscleancargo test -p iggy_connector_sdk --all-featurespassescargo build -p iggy_connector_stdout_sink -p iggy_connector_random_source -p iggy-connectors(macro consumers + runtime rebuild cleanly)