[2/4 messagequeue] Shard MySQL queues by tenant identity - #694
Draft
behinddwalls wants to merge 2 commits into
Draft
[2/4 messagequeue] Shard MySQL queues by tenant identity#694behinddwalls wants to merge 2 commits into
behinddwalls wants to merge 2 commits into
Conversation
behinddwalls
force-pushed
the
preetam/mq-tenant-platform
branch
from
September 8, 2026 06:03
669ef69 to
6ac133e
Compare
## Summary ### Why? Vitess needs a stable vindex that is not the Kafka-style partition key. Isolation has to be an explicit tenant column, with identity carried on publish and consume so two tenants sharing a partition key cannot collide. ### What? - Prefix every queue table and store query with tenant; set queue_offsets PK to (tenant, topic, partition_key, consumer_group). - Use typed (tenant, partitionKey) identities in consumers, gates, and MySQL workers; stop workers when lease discovery cannot confirm ownership. - Require tenant on publish.Message and validate payload queue against message tenant in pipeline controllers (the publish signature change is not compilable without those call sites). - Add ParseRequiredTenants and wire Stovepipe ingest to the configured tenant list so NewIngestController still builds. - Require tenant xor all-tenants on MQ admin list commands; message inspect/delete/requeue take the full (tenant, topic, partition, id) identity. ## Test Plan ✅ `go test` on mysql, publish, consumer, messagequeue identity, service/messagequeue, and start controller Co-authored-by: Cursor <cursoragent@cursor.com>
## Summary ### Why? The CREATE TABLE statements encode two different VARCHAR(255) contracts. A reader opening the SQL should see the byte vs character limits, binary collations, and why tenant stays ASCII. ### What? - Comment ascii/ascii_bin vs utf8mb4/utf8mb4_bin on every queue table, with the full encoding note on queue_messages. Co-authored-by: Cursor <cursoragent@cursor.com>
behinddwalls
force-pushed
the
preetam/mq-tenant-platform
branch
from
September 8, 2026 06:06
6ac133e to
58adda2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why?
Vitess needs a stable vindex that is not the Kafka-style partition key. Isolation has to be an explicit tenant column, with identity carried on publish and consume so two tenants sharing a partition key cannot collide.
What?
Test Plan
✅
go teston mysql, publish, consumer, messagequeue identity, service/messagequeue, and start controllerIssues
Stack