Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the way we check for local/reference table joins in the executor #3280

Merged
merged 4 commits into from
Dec 12, 2019

Conversation

pykello
Copy link
Contributor

@pykello pykello commented Dec 10, 2019

Fixes #3279

Previously we used plannedStmt->releationOids which is the set of relations query depends on to check for the relations "involved" in the query. But this is not correct. For example, citus plans query of the following form in the citus_drop_trigger() to have relationOids = [pg_class, reference_table], so we assumed it is a join between reference table and local table.

SELECT master_unmark_object_distributed(v_obj.classid, v_obj.objid, v_obj.objsubid)

This PR changes the behaviour to use the range table entries instead.

@@ -617,12 +617,22 @@ IsLocalReferenceTableJoinPlan(PlannedStmt *plan)
return false;
}

foreach(oidCell, plan->relationOids)
foreach(rangeTableCell, plan->rtable)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this contain RTEs for the entire query? (incl. CTEs, subqueries)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, standard_planner stores the flattened rte lists in plannedStmt->rtable. set_plan_references concatenates all these lists in glob->finalrtable, and then standard_planner does plannedStmt->rtable = glob->finalrtable.

Added a comment.

if (RelationIsAKnownShard(relationId, onlySearchPath))
if (rangeTableEntry->rtekind == RTE_FUNCTION)
{
return false;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why return false here? What if we a join between a local table, a reference table and a function?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To match distributed_planner.c's IsLocalReferenceTableJoin(). Joins between local, reference, and functions will error out in planner. We can remove that restriction in both planner and executor, but maybe in another PR.

Removing the check here, although I think it won't change any behaviour.

@marcocitus
Copy link
Member

if I try without any works (just coordinator), I get:

postgres=# ALTER TABLE ref ADD COLUMN z int;
WARNING:  could not find any shard placements for shardId 102040
... stuck forever

@marcocitus
Copy link
Member

if I try without any works (just coordinator), I get:

that actually seems to be a separate bug unrelated to having the coordinator in the metadata

@pykello pykello force-pushed the fix_drop_column branch 2 times, most recently from f556d61 to 2c63485 Compare December 11, 2019 00:21
@codecov
Copy link

codecov bot commented Dec 11, 2019

Codecov Report

Merging #3280 into master will increase coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #3280      +/-   ##
==========================================
+ Coverage   92.04%   92.05%   +0.01%     
==========================================
  Files         163      163              
  Lines       32865    32873       +8     
==========================================
+ Hits        30250    30262      +12     
+ Misses       2615     2611       -4

@pykello
Copy link
Contributor Author

pykello commented Dec 11, 2019

if I try without any works (just coordinator), I get:

that actually seems to be a separate bug unrelated to having the coordinator in the metadata

I can look into this tomorrow.

if (RelationIsAKnownShard(relationId, onlySearchPath))
if (rangeTableEntry->rtekind != RTE_RELATION)
{
continue;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since views are RTE_RELATION they are also treated as local tables.

CREATE VIEW v AS SELECT * FROM ref WHERE a = 1;
BEGIN;
SELECT * FROM v JOIN ref ON (v.a = ref.a);
ERROR:  cannot join local tables and reference tables in a transaction block, udf block, or distributed CTE subquery

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed + tests. Also added tests and comments for the function join.

@pykello pykello force-pushed the fix_drop_column branch 4 times, most recently from 52930a4 to 90d2410 Compare December 12, 2019 00:53
@pykello pykello merged commit 0cd1444 into master Dec 12, 2019
@pykello pykello deleted the fix_drop_column branch December 12, 2019 13:13
pykello added a commit that referenced this pull request Dec 12, 2019
pykello pushed a commit that referenced this pull request Dec 12, 2019
@pykello pykello mentioned this pull request Dec 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cannot drop a column from a reference table on the coordinator
2 participants