3.0.0 · pg-sharding spqr · Discussion #2629 · GitHub

Denchick
Jun 19, 2026
Maintainer

We are excited to announce SPQR 3.0.0! All the goals outlined in the SPQR 3.0 Roadmap have been implemented — distributed and reference tables, distributed query support, and two-phase commit for DDL/migrations across shards.

There are no breaking changes between 2.9.x and 3.0.0. We are simply opening a new chapter for SPQR and moving forward.

This release covers all changes from 2.9.9 to 3.0.0, including intermediate patch releases: 2.9.10, 2.9.11, 2.9.12

What's Changed

JOIN Routing & CTE Support

Fix JOIN ON clause routing and add tests by @reshke Fix JOIN on clause routing and add tests #2365
Move tuple routing logic to separate function and add test for JOIN by @reshke Move tuple routing logic to separate function and add test for JOIN i… #2408
Support LEFT and RIGHT join with aux-values multishard by @reshke Support LEFT and RIGHT join with aux-values multishard. #2409
Fix JOIN adjust logic & add tests by @reshke Fix JOIN adjust logic & add tests #2466
Support JOIN CTE adjust for transitive aux values routing by @reshke Support JOIN CTE adjust for transitive aux values routing. #2467
Support multiple AUX CTE with one common ancestor for JOIN adjust by @reshke Support multiple AUX CTE with one common ancestor for JOIN adjust fea… #2468
Fix CTE alias referencing by @reshke Fix CTE alias referencing #2415
Fix relation alias resolve and add regress test by @reshke Fix relation alias resolve and add regress test #2543

Extended Protocol & Multi-Portal

Partially support multi-portal binding by @reshke Partially support multi-portal binding #2519
Stub implementation for portal suspension by @reshke Stub implementation for portal suspension #2522
Use ProcessOneMsgCarefully for Describe by @reshke Use ProcessOneMsgCarefully for Describe #2442
Remove custom processing in Execute by @reshke Remove custom processing in Execute #2443
Fix pgproto message copy for simple query & parse by @reshke Fix pgproto message copy for simple query & parse #2492
Greasify protocol negotiation in startup processing by @reshke Greasify protocol negotiation in startup processing #2608
Respect result format code in virtual pg_is_in_recovery() by @reshke Respect result format code in virtual pg_is_in_recovery() #2517

Two-Phase Commit (2PC)

Simple happy-path error recovery in two phase commit by @reshke Simple happy-path error recovery in two phase commit #2480
Fix transaction deploy on ROLLBACK by @reshke Fix transaction deploy on ROLLBACK #2481
Support client feedback for 2pc commit flow by @reshke Support client feedback for 2pc commit flow #2531
Add __spqr__set_next_2pc_gid and use it in tests by @reshke Add __spqr__set_next_2pc_gid and use it in tests #2536
Fix two phase recovery utility function by @reshke Fix two phase recovery utility function #2574
Fix 2pc tx state recovery by @reshke Fix 2pc tx state recovery #2578
Add isolation tests for 2pc failure recovery by @reshke Add isolation test for concurrent transaction deploy and backend shut… #2482 Add isolation test for happy-path error recovery in case of 2pc failure #2485

Implicit Transactions

Support implicit tx blocks by @reshke Support implicit tx blocks #2557
Add notice message for router implicit tx feature by @reshke Add notice message for router implicit tx feature #2561
Fix implicit transaction error recovery flow by @reshke Fix implicit transaction error recovery flow #2564

Auto-Linearize for Multi-Shard DML

Auto-linearize hazard multi-shard upserts by @reshke Auto-linearize hazard multi-shard upserts #2512
Fix auto-linearize on INSERT INTO .. RETURNING by @reshke Fix auto-linearize on INSERT INTO .. RETURNING #2527
Reimplement dispatch linearize as plan hinting rather than GUC injection by @reshke Reimplement dispatch linearize as plan hinting rather than GUC injection #2560
Support correct command complete tag in multishard UPDATE by @reshke Support correct CC tag in multishard UPDATE #2618 Support correct CC tag in multishard UPDATE in xproto #2622
Fix multishard insert command complete tag by @reshke Fix multishard insert command complete tag #2626

Data Transfer & Balancer

Allow context cancellation for data transfer workers by @reshke Allow context cancellation for data transfer workers #2386
Allow to cancel data transfer task in any execution phase by @reshke Allow to cancel data transfer task in any execution phase #2390
Cancel data move task only on IMMEDIATE flag by @reshke Cancel data move task only on IMMEDIATE flag #2456
Recheck key range status for in-flight data chunk by @reshke Recheck key range status for in-flight data chunk #2569
GUCify in-flight data access option by @reshke GUCify in-flight data access option #2571
Do not perform parallel transactions on both shards in copyData by @EinKrebs Do not perform parallel transactions on both shards in copyData #2627
Trace all data transfer goroutines in coordinator by @reshke Trace all data transfer goroutines in coordinator #2385

Metadata & Key Range Management

Metadata version management basics by @EinKrebs Metadata version management basics #2395
Implement router rebootstrap through SQL by @EinKrebs Implement router rebootstrap through SQL #2398
Update key range metadata in spqrguard by @EinKrebs Update key range metadata in spqrguard #2424
Maintain key range meta on rename by @EinKrebs Maintain key range meta on rename #2594
Separate spqrguard meta update from external writes lock in config by @EinKrebs Separate spqrguard meta update from external writes lock in config #2611
Add a check for an exact key match in qdb/etcd by @Pavloxes Added a check for an exact key match in qdb/etcd in new version #2552

Console & SQL Enhancements

HELP for register/unregister routers, misc commands by @Denchick HELP register/unregister routers #2367 HELP misc #2371
Enforce 80-char line limit in HELP by @Denchick enforce 80-char line limit in HELP #2453
Deprecate ADD keyword, use CREATE instead by @Denchick Deprecate ADD keyword, use CREATE instead #2038
Implement LOCAL & GLOBAL show kinds by @reshke Implement LOCAL & GLOBAL show kinds #2440
Support multistatement SQL in console by @reshke Support multistatement SQL in console #2541
Add ALTER SYSTEM ROTATE for spqr-console log rotation by @reshke Add ALTER SYSTEM ROTATE for spqr-console log rotation #2558
Add SHOW SHARDS_EXTENDED command by @diPhantxm Add SHOW SHARDS_EXTENDED command to spqr-console #2562
Support ALTER SHARD in router with coordinator by @diPhantxm Support ALTER SHARD in router with coordinator #2428 More tests for ALTER SHARD and fix #2445
Add discard count column to SHOW pools by @reshke Add discard count column to SHOW pools #2615
Add server_lifetime setting to backend rules by @reshke Add server_lifetime setting to backend rules #2616
Improve shard-not-found errors in console by @Minetoff Improve shard-not-found errors in console. #2175
SQL part of GRANT & alter system re-bootstrap by @reshke SQL part of GRANT & alter system re-bootstrap #2381

Monitoring & Observability

Prometheus metrics for router (MVP) by @xelavopelk Prometheus metrics for router (MVP) #2497
Reimplement RPS counting as part of statistics package by @reshke Reimplement RPS counting as part of statistics package #2475
Fix router tx/shard statement quantiles by @reshke Fix router tx/shard statement quantiles #2493
Add simple query log time on INFO level by @reshke Add simple query log time on INFO level #2532
Add and use TRACE severity for some debug messages by @reshke Add and use TRACE serverity for some debug messages #2533
Write log asynchronously (router only) by @reshke Write log asynchronously #2491 Use async logger only in router #2501
Downgrade unmatched TSA host log level from Error to Debug by @Denchick Downgrade unmatched TSA host log level from Error to Debug #2488

spqr-monitor

Initial implementation for spqr-monitor by @EinKrebs Initial implementation for spqr-monitor #2566

Shard Management & Networking

Handle hosts with AZ in shard hosts validation by @diPhantxm Handle hosts with AZ in shard hosts validation #2486
Pass hosts' AZ to distributed mgr by @diPhantxm pass hosts' AZ to distributed mgr #2498
Send host list with AZ on router sync by @diPhantxm Send host list with AZ on router sync #2579
Add gRPC validation rules by @diPhantxm Add gRPC Validation rules #2450
Fix memqdb rebootstrap losing shards by @diPhantxm Fix memqdb rebootstrap losing shards #2441
Make shard replicas synchronous by @EinKrebs Make shard replicas synchronous #2515
Change synchronous_commit to remote_apply on shards by @EinKrebs Change synchronous_commit to remote_apply on shards #2521
Pin client catalog queries to dedicated shard on setup by @reshke Pin client catalog queires to dedicated shard on setup #2609

Error Reporting

Modernize coding around error reporting by @reshke Modernize coding around error reporting #2372
Switch errComplexQuery to use spqrerror infrastructure by @reshke Switch errComplexQuery to use spqrerror infrastructure #2414
Use SPQR-specific error format for general error report by @reshke Use SPQR spesific error format for general error report #2433
Report key range id as errdetail by @reshke Report key range id as errdetail #2423
Set QUERY field in SPQR-specific error for simple proto by @reshke Set QUERY field in SPQR-specific error for simple proto #2535

Configuration

Support UUID HASH column type in distributions by @Denchick Support UUID HASH column type in distributions #2588
Support disabling idle_in_transaction_session_timeout by @EinKrebs Support disabling idle_in_transaction_session_timeout #2617
Support no-op distribution column update by @reshke Support no-op distribution column update #2621
Cache schema in relay and share between executions by @reshke Cache schema in relay and share between executions #2524

Stability & Panic Fixes

Fix panic in SHOW relation & refactor by @EinKrebs Fix panic in "SHOW relation" & refactor #2407
Fix panic in multishard conversion by @reshke Fix panic in multishard convertion #2526
Fix router startup panic by @xelavopelk Fix router startup panic #2494
Fix panic: unexpected proto version message size by @diPhantxm Fix panic: unexpected proto version message size #2537
Fix panic & release goroutines on route obsoletion by @reshke Fix panic & release goroutines on route obsoletion #2577
Fix/out of range binds panic by @Ledatu Fix/out of range binds panic #2556
Remove panic from stat collector by @reshke Remove panic from stat collector #2476
Make ICP panic action to be os.Exit() by @reshke Make ICP panic action to be os.Exit() #2499

Code Quality & Linting

Fix var-naming linting issues, parts 1–4 by @Denchick Fix var-naming linting issues, part 1 #2378 Fix var-naming linting issues, part 2 #2420 Fix var-naming linting issues, part 3 #2427 Fix var-naming lint issues, part 4 #2431
Fix unchecked-type-assertion, parts 1–2 by @Denchick fix unchecked-type-assertion, part 1 #2392 fix unchecked-type-assertion, part 2 #2394
Fix unexported-naming, receiver-naming, var-declaration, unnecessary-if lint issues by @Denchick fix unexported-naming lint issues #2377 fix receiver-naming linting issues #2412 fix var-declaration lint issues #2432 fix unnecessary-if lint issues #2402
Replace golang.org/x/exp/slices with stdlib slices by @Denchick Replace golang.org/x/exp/slices with stdlib slices #2473
Remove dead code by @reshke Remove dead code #2404 Remove dead code part two #2477 Drop dead code #2539
Remove ShardMapping, use TopologyManager instead by @reshke Remove ShardMapping, use TopologyManager instead #2437

Tests

Extended protocol tests: Copy, Close, Execute(MaxRows), simple-in-extended, BEGIN+extended and more by @rkhapov proto_test.go: expand Copy in xproto test #2379 proto_test.go: Close for unexisting stmt #2387 split proto tests #2388 copy_proto_test.go: split to several tests #2397 xproto_test.go: make tests more idempotent #2406 xproto_test.go: Execute(MaxRows) test #2416 xproto_test.go: fix Execute(MaxRows) #2417 xproto_test.go: simple in the middle of extended #2438 proto_test.go: additional tests #2542 xproto: test simple BEGIN + extended #2582 xproto_test.go: fix compatibility with vanilla #2583
Tests: Oid Field Description by @saneczkab Tests: Oid Field Description #2425
Add unit test CmpRangesLessStringDeprecated by @Vinter-mach Add unit test CmpRangesLessStringDeprecated #2502
Add unit test: WHERE MatchRow by @NeSawer Add unit test: WHERE MatchRow #2540
Mock ClientInfo, RuleRouter and test ListClients, ListBackendConnections, ListPools by @AnastasiaKostareva mock ClientInfo, RuleRouter and tests ListClients, ListBackendConnections, ListPools #2451
Fix flaky feature tests by @xelavopelk Fix flacky coordinator.feature test #2439 Fix flacky tls_shard.feature test #2444 fix flacky coordinator.feature and proxy_console.feature tests #2458 fix proxy_console.feature test #2464
Fix feature test flakes by isolating auth fixtures by @Denchick fix feature test flakes by isolating auth fixtures #2573

Documentation

Add manual resharding guide by @singl3focus docs: add manual resharding guide #2435
Fix LOCK KEY RANGE behavior description by @Denchick docs: fix LOCK KEY RANGE behavior description (error, not wait) #2448
Fix typo in get_started guide by @kolzuk doсs: fix typo in get_started guide #2586
Add isolation tests README by @Denchick docs: add isolation tests README #2603

DevOps & CI

Use content hash for shard image cache by @Denchick ci: use content hash for shard image cache #2452
Fix mismatched hashFiles inputs between feature_prepare and feature runs by @Denchick ci: fix mismatched hashFiles inputs between feature_prepare and featu… #2465
Fix CI unhealthy container log collection by @Denchick Fix CI unhealthy container log collection #2597
Fix regress test teardown in schema_qual.sql by @Denchick Fix regress test teardown in schema_qual.sql #2592
Bump PostgreSQL version by @reshke Bump PostgreSQL version #2546
Bumps by @dependabot Bump github.com/Azure/go-ntlmssp from 0.1.0 to 0.1.1 #2373 Bump google.golang.org/grpc from 1.80.0 to 1.81.0 #2469 Bump go.etcd.io/etcd/client/v3 from 3.6.10 to 3.6.11 #2470 Bump golang.org/x/crypto from 0.50.0 to 0.51.0 #2511 Bump google.golang.org/grpc from 1.81.0 to 1.81.1 #2530 Bump golang.org/x/crypto from 0.51.0 to 0.52.0 #2565 Bump go.etcd.io/etcd/client/v3 from 3.6.11 to 3.6.12 #2589 Bump github.com/jackc/pgx/v5 from 5.9.2 to 5.10.0 #2593 Bump golang.org/x/sync from 0.20.0 to 0.21.0 #2602 Bump golang.org/x/crypto from 0.52.0 to 0.53.0 #2607

New Contributors

@saneczkab made their first contribution in Tests: Oid Field Description #2425
@singl3focus made their first contribution in docs: add manual resharding guide #2435
@Vinter-mach made their first contribution in Add unit test CmpRangesLessStringDeprecated #2502
@NeSawer made their first contribution in Add unit test: WHERE MatchRow #2540
@kolzuk made their first contribution in doсs: fix typo in get_started guide #2586
@Pavloxes made their first contribution in Added a check for an exact key match in qdb/etcd in new version #2552

Full Changelog: 2.9.9...3.0.0

This discussion was created from the release 3.0.0.

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment