feat: improve event transport throughput#635
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #635 +/- ##
==========================================
+ Coverage 75.85% 76.04% +0.19%
==========================================
Files 473 473
Lines 65285 66107 +822
Branches 9731 9841 +110
==========================================
+ Hits 49522 50273 +751
- Misses 12504 12542 +38
- Partials 3259 3292 +33
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
e9bd451 to
c9dbb2a
Compare
4e9d86b to
151ff07
Compare
f020dd4 to
05f00a1
Compare
cofin
added a commit
that referenced
this pull request
Jul 10, 2026
## Summary - align bounded row-stream cleanup and transaction behavior across adapters - consolidate Arrow fallback behavior and adapter contract cleanup - define canonical event transports: `notify`, `notify_queue`, `poll_queue`, `aq`, and `txeventq` - add typed adapter event configuration and accurate durable-delivery documentation ## Review fixes - context-manager failures roll back and cleanup `TypeError` is not retried - legacy duck-typed stream sources with `close()` still clean up correctly - mysql-connector early close consumes unread rows without replacing an active transaction session - retired event names fail explicitly with canonical migration guidance rather than silently selecting polling - adapter `events_backend` features are honored when extension configuration does not override them - native event fixtures remain collected and Python 3.14 type introspection remains supported ## Stack This is the base PR for #635, which contains listener lifecycle, batching, polling/recovery, and final transport documentation. ## Validation - focused stream, Arrow, event, mysql-connector, and configuration tests - Ruff, mypy, and pyright - full GitHub unit/integration matrix on Python 3.10-3.14 - docs, source distribution, pure wheel, MyPyC wheel, and package integrity checks All GitHub checks are green.
ccfd08e to
85cd58c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
publish_many()with ordered IDs, explicit grouped transactions, and an ordered fallback{marker_id, batch_size}wakeup marker per channel atomicallyevent_poll_intervalreconciliation and immediately drain every recovered rowSQLSpecChannelsBackend.publish_many(data, channels)for consumer integrationIntegration
This PR targeted
maindirectly after #633 and #634 merged. Its event-only commits were rebased onto merge commitff0b8cb51, so the final branch validates the consolidated C4 contracts and lazy-typing implementation together.Correctness notes
Durable rows are the source of truth. Native markers are wakeup hints, not batch envelopes. Row insertion and marker publication share one explicit transaction, marker publication failure rolls back the rows, duplicate markers are suppressed, and one reconciliation timeout enters drain mode until the queue is empty. Durable transports remain competing-consumer queues rather than browser fan-out.
JSON payloads and metadata remain native values through batch parameter preparation so adapter-specific encoders receive the same input shape as single-event publication.
Validation