Skip to content

Use direct PublisherClient in BQ shadow sink#9

Merged
cmttt merged 1 commit intomainfrom
ls/fix-bq-sink-publisher
Mar 31, 2026
Merged

Use direct PublisherClient in BQ shadow sink#9
cmttt merged 1 commit intomainfrom
ls/fix-bq-sink-publisher

Conversation

@cmttt
Copy link
Copy Markdown
Owner

@cmttt cmttt commented Mar 31, 2026

BatchPubsubPublisherClient has a class-level threading.Lock that causes 'cannot release un-acquired lock' when released from a background callback thread. Replace with direct PublisherClient (batch_size=1, no background threads). Also removes dependency on osprey.worker.lib.publisher.

@cmttt cmttt force-pushed the ls/fix-bq-sink-publisher branch 2 times, most recently from 424edcf to a678938 Compare March 31, 2026 21:11
…ow sink

New AsyncPubSubPublisher:
- asyncio.Queue buffer with background flush task
- Batches up to 250 messages or flushes after 1s
- Flush runs sync publishes in run_in_executor
- Graceful shutdown flushes remaining messages
- No threading locks, no gevent, no background threads

Removed BQ shadow sink from osprey — it imports from discord_smite
and belongs in the Discord repo (discord_smite/osprey_async_plugins).
@cmttt cmttt force-pushed the ls/fix-bq-sink-publisher branch from a678938 to 5543e07 Compare March 31, 2026 21:27
@cmttt cmttt merged commit 313d620 into main Mar 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant