Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create index fails if hypertable has foreign table chunk #4899

Merged
merged 1 commit into from Feb 27, 2023

Conversation

gayyappan
Copy link
Contributor

@gayyappan gayyappan commented Oct 28, 2022

We cannot create indexes on foreign tables. This PR modifies process_index_chunk to skip OSM chunks

Disable-check: force-changelog-changed

@codecov
Copy link

codecov bot commented Oct 28, 2022

Codecov Report

Merging #4899 (32589da) into main (152ef02) will decrease coverage by 0.15%.
The diff coverage is 92.85%.

❗ Current head 32589da differs from pull request most recent head daa4286. Consider uploading reports for the commit daa4286 to get more accurate results

@@            Coverage Diff             @@
##             main    #4899      +/-   ##
==========================================
- Coverage   90.93%   90.79%   -0.15%     
==========================================
  Files         225      225              
  Lines       52327    51533     -794     
==========================================
- Hits        47586    46787     -799     
- Misses       4741     4746       +5     
Impacted Files Coverage Δ
src/process_utility.c 94.46% <92.85%> (-0.01%) ⬇️
tsl/src/nodes/frozen_chunk_dml/frozen_chunk_dml.c 0.00% <0.00%> (-84.45%) ⬇️
tsl/test/src/test_chunk_stats.c 48.14% <0.00%> (-44.45%) ⬇️
tsl/src/nodes/compress_dml/compress_dml.c 84.78% <0.00%> (-13.05%) ⬇️
src/chunk.c 88.91% <0.00%> (-4.84%) ⬇️
tsl/src/chunk.c 89.67% <0.00%> (-4.72%) ⬇️
src/bgw/scheduler.c 84.39% <0.00%> (-4.10%) ⬇️
tsl/src/partialize_finalize.c 92.72% <0.00%> (-3.23%) ⬇️
src/chunk_constraint.c 90.44% <0.00%> (-3.12%) ⬇️
src/loader/bgw_message_queue.c 86.36% <0.00%> (-2.85%) ⬇️
... and 31 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@gayyappan gayyappan marked this pull request as ready for review October 31, 2022 11:50
@gayyappan gayyappan requested review from fabriziomello and removed request for svenklemm October 31, 2022 11:51
@@ -394,6 +394,14 @@ FROM chunk_view
WHERE hypertable_name = 'hyper_constr'
ORDER BY chunk_name;

--Utility functions -check index creation on hypertable with OSM chunk
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO will be nice if you can improve a bit the comments here for people understand that we're skipping {CREATE|DROP} INDEX on OSM chunks...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added more comments here.

@@ -2494,6 +2495,14 @@ process_index_chunk_multitransaction(int32 hypertable_id, Oid chunk_relid, void
}
#endif

chunk = ts_chunk_get_by_relid(chunk_relid, true);
if (IS_OSM_CHUNK(chunk)) /*cannot create index on foreign OSM chunk */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about add a NOTICE message where about skipping?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, any notice/error should be managed by OSM. This is an internal chunk and notice/error would not make sense to the end user.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By moving chunk = ts_chunk_get_by_relid(chunk_relid, true); above you're breaking the lock semantics described below:

...
	 * Hold a lock on the hypertable index, and the chunk to prevent
	 * from being altered. Since we use the same relids across transactions,
...

Copy link
Contributor Author

@gayyappan gayyappan Oct 31, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching the comment.I don't quite follow what it means

2514      * Hold a lock on the hypertable index, and the chunk to prevent
2515      * from being altered. Since we use the same relids across transactions,
2516      * there is a potential issue if the id gets reassigned between one
2517      * sub-transaction and the next. CLUSTER has a similar issue.

How does the id for a chunk get reassigned? Is this referring to chunk relid or chunk->fd.id or something else?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really know.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems this is to prevent a concurrent session alter the same object (alter/drop columns, add/drop indexes, etc). Will be better @gayyappan to move your check and skip to after table_open and index_open

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gayyappan is there any problems with moving this block back into the lock-protected section?

Patch: https://gist.github.com/zilder/a9b2b61af07361ae6a938b75ae516fb2

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it is ok to move the check after the table_open. This code block will not work as-is. Note that we should clear state correctly before exiting from the function. Probably easiest with an if-else stmt.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved the code around to address the problem described by mfundul.

We cannot create indexes on foreign tables. This PR modifies
process_index_chunk to skip OSM chunks
Copy link
Contributor

@fabriziomello fabriziomello left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gayyappan gayyappan enabled auto-merge (rebase) February 27, 2023 16:46
@gayyappan gayyappan enabled auto-merge (rebase) February 27, 2023 17:43
@gayyappan gayyappan merged commit 2f7e043 into timescale:main Feb 27, 2023
@gayyappan gayyappan added this to the TimescaleDB 2.10.1 milestone Mar 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants