Fix inserting to pg_dist_object for queries from other nodes #7402
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #7402 +/- ##
=======================================
Coverage 89.61% 89.61%
=======================================
Files 280 280
Lines 60310 60318 +8
Branches 7507 7509 +2
=======================================
+ Hits 54044 54052 +8
+ Misses 4113 4111 -2
- Partials 2153 2155 +2 |
halilozanakgul
force-pushed
the
fix_other_db_pg_dist_obj_update
branch
from
f5e9062
to
a202181
Compare
JelteF
reviewed
Jan 4, 2024
Comment on lines
8
to
10
| c.sql("SELECT citus_set_coordinator_host('localhost')") | ||
|
|
There was a problem hiding this comment.
Is this actually necessary? Afaict CitusCluster its __init__ function already does this. If that doesn't work for some reason we should fix it there instead of working around the problem here.
i.e. this should be removed afaict.
JelteF
approved these changes
Jan 4, 2024
| @@ -60,14 +73,23 @@ def test_main_commited_outer_not_yet(cluster): | |||
| int(role_after_commit) == 1 | |||
| ), "role is not created during recovery despite committing" | |||
|
|
|||
| # check that the user is on pg_dist_object on the coordinator after transaction recovery | |||
There was a problem hiding this comment.
Suggested change
| # check that the user is on pg_dist_object on the coordinator after transaction recovery | |
| # check that the user is in pg_dist_object on the coordinator after transaction recovery |
| assert ( | ||
| int(pdo_coordinator_after_commit) == 1 | ||
| ), "role is not on pg_dist_object on coordinator after recovery despite committing" | ||
|
|
||
| # check that the user is on pg_dist_object on the worker after transaction recovery |
There was a problem hiding this comment.
Suggested change
| # check that the user is on pg_dist_object on the worker after transaction recovery | |
| # check that the user is in pg_dist_object on the worker after transaction recovery |
| assert ( | ||
| int(pdo_coordinator_after_recovery) == 0 | ||
| ), "role is on pg_dist_object on coordinator after recovery despite aborting" | ||
|
|
||
| # check that the user is not on pg_dist_object on the worker after transaction recovery |
There was a problem hiding this comment.
Suggested change
| # check that the user is not on pg_dist_object on the worker after transaction recovery | |
| # check that the user is not in pg_dist_object on the worker after transaction recovery |
| @@ -142,13 +181,22 @@ def test_main_commited_outer_aborted(cluster): | |||
| int(role_after_recovery) == 0 | |||
| ), "role is created during recovery despite aborting" | |||
|
|
|||
| # check that the user is not on pg_dist_object on the coordinator after transaction recovery | |||
There was a problem hiding this comment.
Suggested change
| # check that the user is not on pg_dist_object on the coordinator after transaction recovery | |
| # check that the user is not in pg_dist_object on the coordinator after transaction recovery |
JelteF
reviewed
Jan 4, 2024
| @@ -80,7 +81,8 @@ mark_object_distributed(PG_FUNCTION_ARGS) | |||
| Oid objectId = PG_GETARG_OID(2); | |||
| ObjectAddress *objectAddress = palloc0(sizeof(ObjectAddress)); | |||
| ObjectAddressSet(*objectAddress, classId, objectId); | |||
| MarkObjectDistributedWithName(objectAddress, objectName); | |||
| bool useConnectionForLocalQuery = true; | |||
There was a problem hiding this comment.
I think it's useful to say why we want true here (I pretty much copy pasted the one you had in the other spot, feel free to change)
Suggested change
| bool useConnectionForLocalQuery = true; | |
| /* | |
| * This function is called when a query is run from a Citus non-main database. We need to insert into pg_dist_object | |
| * over a connection to make sure 2PC still works. | |
| */ | |
| bool useConnectionForLocalQuery = true; |
halilozanakgul
force-pushed
the
fix_other_db_pg_dist_obj_update
branch
from
58ee22f
to
b67560a
Compare
JelteF
reviewed
Jan 11, 2024
Comment on lines
8
to
9
| c.sql("ALTER SYSTEM SET citus.local_hostname TO '127.0.0.1'") | ||
| c.sql("SELECT pg_reload_conf()") |
There was a problem hiding this comment.
This should not be necessary. In production environments it's expected that citus.local_hostname is not the same as the value in pg_dist_node. So if the tests require that to pass, then that's a big problem (because that means they would fail in production environments).
JelteF
approved these changes
Jan 11, 2024
JelteF
force-pushed
the
fix_other_db_pg_dist_obj_update
branch
from
73ff86f
to
e55d8fe
Compare
JelteF
approved these changes
Jan 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Running a query from a Citus non-main database that inserts to pg_dist_object requires a new connection to the main database itself. This PR adds that connection to the main database.