Merge remote-tracking branch 'origin/main' into feature/ddl-from-any-node-schema-based-sharding#8521
Merged
onurctirtir merged 8 commits intofeature/ddl-from-any-node-schema-based-shardingfrom Mar 17, 2026
Conversation
See the linked PR for more details.
…8480) IsTenantSchema checks pg_dist_schema when the version check is disabled. Previously, IsTenantSchema() had a guard that returned false whenever citus.enable_version_checks was off. This was added to protect against accessing pg_dist_schema in multi_extension tests that install old Citus versions (pre-12.0) where the table doesn't exist. However, the guard had an unintended side effect: it completely disabled schema-based sharding during mixed-version testing (CITUSVERSION/N1MODE), even when no actual version mismatch existed. This caused 5 tests to fail (single_node, schema_based_sharding, citus_schema_distribute_undistribute, citus_schema_move, local_shard_utility_command_execution) because tables created in tenant schemas were not being distributed. The root cause was an asymmetry: CREATE SCHEMA correctly registered the tenant schema in pg_dist_schema (ShouldUseSchemaBasedSharding has no version check guard), but CREATE TABLE in that schema silently fell through to a local table because IsTenantSchema returned false. CheckCitusVersion(). When version checks are disabled, CheckCitusVersion() returns true unconditionally, which is safe because: - pg_dist_schema exists in all supported Citus versions (>= 12.0) - mixed-version tests never install versions old enough to lack it - multi_extension (which installs pre-12.0 versions) is not run in mixed-version scenarios
…n_timeout on metadata workers (#8484) ### Description When performing a shard move using block_writes transfer mode (either directly via citus_move_shard_placement or through the background rebalancer), the operation can fail with: ``` ERROR: terminating connection due to idle-in-transaction timeout CONTEXT: while executing command on <worker_host>:<worker_port> ``` The failing worker is a metadata worker that is neither the source nor the target of the shard move. ### Root Cause LockShardListMetadataOnWorkers() opens coordinated transactions on all metadata workers to acquire advisory shard metadata locks via SELECT lock_shard_metadata(...). These transactions remain open until the entire shard move completes and the coordinated transaction commits. In block_writes mode, the data copy phase (CopyShardsToNode) runs synchronously between the source and target workers. Metadata workers not involved in the copy have no commands to execute and their connections sit completely idle-in-transaction for the entire duration of the data copy. For large shards, the copy can take significantly longer than common idle_in_transaction_session_timeout values, When the timeout fires on an uninvolved worker, PostgreSQL terminates the connection, causing the shard move to fail. This also affects shard splits, since they follow the same code path through LockShardListMetadataOnWorkers. ### Fix LockShardListMetadataOnWorkers() should send SET LOCAL idle_in_transaction_session_timeout = 0 on each metadata worker connection before acquiring the locks. SET LOCAL scopes the change to the current transaction only, so normal sessions on the workers are unaffected.
## Summary This PR addresses all currently open Dependabot alerts in this repository by updating vulnerable Python dependencies in both mirrored regression-test environments: - src/test/regress/Pipfile / Pipfile.lock - .devcontainer/src/test/regress/Pipfile / Pipfile.lock ## Updated dependencies - cryptography: 44.0.3 -> 46.0.5 (patched: >=46.0.5) - Werkzeug: 3.1.4 -> 3.1.5 (patched: >=3.1.5) - filelock: resolved to 3.25.0 (patched: >=3.20.3) - pyasn1: resolved to 0.6.2 (patched: >=0.6.2) ## Alerts covered Closes Dependabot alerts: #98, #99, #100, #101, #102, #103, #104, #105, #106, #107, #108, #109. ## Notes Lockfiles were regenerated with pipenv lock in both directories to ensure consistent, hashed resolution.
DESCRIPTION: Support extended stats creation on expressions CREATE STATISTICS today gets pushed to the shards only if the statistics is on a column expression. However, PG14 added support for expressions (opExpr, FuncExprs) etc. This change adds support for it under a GUC. This reuses the same logic as CHECK constraints to parse, transform, and deparse the exprs and add it to the statistics call.
DESCRIPTION: Add support for aggregates with an internal stype Citus has historically required custom aggregates to not have an internal stype except for specific internal aggregates. This has led to a number of workarounds to get performance and custom aggregates working with distributed tables. This change removes that restriction by mirroring Postgres's use of the SERIALFUNC and DESERIALFUNC to roundtrip state for aggregates' internal stype metadata between workers and coordinators allowing more natural use of custom aggregates in Citus.
…node-schema-based-sharding
onurctirtir
merged commit 44d066e
into
feature/ddl-from-any-node-schema-based-sharding
29 checks passed
onurctirtir
deleted the
sync-main-to-feature/ddl-from-any-node-schema-based-sharding
branch
Codecov Report❌ Patch coverage is ❌ Your project status has failed because the head coverage (84.47%) is below the target coverage (87.50%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## feature/ddl-from-any-node-schema-based-sharding #8521 +/- ##
===================================================================================
- Coverage 88.87% 84.47% -4.41%
===================================================================================
Files 286 286
Lines 63638 63715 +77
Branches 7987 8001 +14
===================================================================================
- Hits 56560 53823 -2737
- Misses 4775 7366 +2591
- Partials 2303 2526 +223 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.