-
-
Notifications
You must be signed in to change notification settings - Fork 128
feat(connection): Improve connection handling #471
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Pull Request Test Coverage Report for Build 19697831375Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 19676588258Details
💛 - Coveralls |
iambriccardo
commented
Nov 26, 2025
| // PRIORITY 2: Process incoming replication messages from Postgres. | ||
| // This is the primary data flow, converts replication protocol messages | ||
| // into typed events and accumulates them into batches for efficient processing. | ||
| Some(message) = logical_replication_stream.next() => { |
Contributor
Author
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was an error from our first implementation. After digging into the code, when the channel is closed, the stream returns None instead of erroring, thus this caused the pipeline to stall in some cases.
srcCraftsman
approved these changes
Nov 26, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes a connection handling issue where we did not handle the
Nonecase from the replication stream. This caused the pipeline to stall when the connection was not gracefully closed since the library returnsNonewhen the connection is closed:This way now the system handles both the case when Postgres sends the message via the connection (e.g., an error) and when the connection is actually closed.
All of this was replicated locally:
pg_terminate_backendfor the termination of the stream via the error (was already handled).docker killfor the abrupt termination which was just closing the connection (was not handled before and now it is).