Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce a FDW option to mark reference tables #5189

Merged
merged 1 commit into from Jan 18, 2023

Conversation

jnidzwetzki
Copy link
Contributor

With this patch, the ability to mark reference tables (tables that exist on all data nodes of a multi-node installation) via an FDW option has been added.

Note: This patch is part of a series of patches that allows join pushdowns in the multi-node code. This commit also introduces the regression test files for the join pushdowns. The test output is currently the same for all PG versions. However, this will change when the test cases for the join pushdown are added.

@jnidzwetzki jnidzwetzki self-assigned this Jan 17, 2023
@jnidzwetzki jnidzwetzki marked this pull request as ready for review January 17, 2023 07:53
@codecov
Copy link

codecov bot commented Jan 17, 2023

Codecov Report

Merging #5189 (9958d67) into main (9a2cbe3) will decrease coverage by 0.02%.
The diff coverage is 81.48%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #5189      +/-   ##
==========================================
- Coverage   89.43%   89.41%   -0.02%     
==========================================
  Files         225      225              
  Lines       51693    51719      +26     
==========================================
+ Hits        46230    46245      +15     
- Misses       5463     5474      +11     
Impacted Files Coverage Δ
tsl/src/fdw/relinfo.c 95.87% <25.00%> (-1.50%) ⬇️
tsl/src/fdw/option.c 55.23% <91.30%> (+12.55%) ⬆️
src/bgw/scheduler.c 83.88% <0.00%> (-1.54%) ⬇️
src/loader/bgw_message_queue.c 88.63% <0.00%> (-0.57%) ⬇️
tsl/src/reorder.c 85.87% <0.00%> (-0.23%) ⬇️
tsl/src/bgw_policy/job.c 87.28% <0.00%> (-0.05%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9a2cbe3...9958d67. Read the comment docs.

if (rel->rd_rel->relkind != RELKIND_RELATION)
ereport(ERROR,
(errcode(ERRCODE_INVALID_TABLE_DEFINITION),
errmsg("table \"%s\" is not a relation. Only relations can be used as "
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The message should be the other way around? "relation is not an ordinary table".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akuzm Thanks, it is changed.

@@ -78,6 +78,13 @@ apply_fdw_and_server_options(TsFdwRelInfo *fpinfo)
option_extract_extension_list(defGetString(def), false));
else if (strcmp(def->defname, "fetch_size") == 0)
fpinfo->fetch_size = strtol(defGetString(def), NULL, 10);
else if (strcmp(def->defname, "join_reference_tables") == 0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the name of the parameter, I'd draw attention to the fact that this is for replicated reference tables. E.g. replicated_reference_tables. Reference tables are not necessarily replicated, although we came to imply this in our terminology, so it might be confusing for the new users.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A good point. I started a thread in Slack regarding the name.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed according to the discussion in Slack.

@@ -0,0 +1,173 @@
-- This file and its contents are licensed under the Timescale License.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to have a more specific name for the test, like dist_ref_table_join.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The files are renamed to dist_ref_table_join as suggested.

Copy link
Contributor

@erimatnor erimatnor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few nits

/* syntax error in name list */
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("parameter \"%s\" must be a list of reference table names",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
errmsg("parameter \"%s\" must be a list of reference table names",
errmsg("parameter \"%s\" must be a comma-separated list of reference table names",

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@erimatnor Thanks for the review. The error messages are changed in the current version of the PR.

{
ereport(ERROR,
(errcode(ERRCODE_INVALID_TABLE_DEFINITION),
errmsg("table \"%s\" does not exist.", tablename)));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
errmsg("table \"%s\" does not exist.", tablename)));
errmsg("table \"%s\" does not exist", tablename)));

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed

ereport(ERROR,
(errcode(ERRCODE_INVALID_TABLE_DEFINITION),
errmsg("table \"%s\" is not a relation. Only relations can be used as "
"reference tables.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"reference tables.",
"reference tables",

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also changed

@jnidzwetzki jnidzwetzki force-pushed the mn_join_fdw_option branch 2 times, most recently from 0a5057c to 2f4f716 Compare January 18, 2023 13:17
@jnidzwetzki jnidzwetzki enabled auto-merge (rebase) January 18, 2023 13:18
With this patch, the ability to mark reference tables (tables that exist
on all data nodes of a multi-node installation) via an FDW option has
been added.
@jnidzwetzki jnidzwetzki merged commit 19065bb into timescale:main Jan 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants