Fix chunk exclusion for space partitions in SELECT FOR UPDATE queries #4685
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## main #4685 +/- ##
==========================================
- Coverage 90.85% 90.82% -0.04%
==========================================
Files 224 224
Lines 42326 42344 +18
==========================================
+ Hits 38455 38457 +2
- Misses 3871 3887 +16
Continue to review full report at Codecov.
|
akuzm
approved these changes
Sep 9, 2022
svenklemm
requested review from
fabriziomello,
gayyappan,
jnidzwetzki,
konskov and
mfundul
jnidzwetzki
approved these changes
Sep 9, 2022
svenklemm
force-pushed
the
rowmark
branch
3 times, most recently
from
5e06d37
to
fb52a76
Compare
Since we do not use our own hypertable expansion for SELECT FOR UPDATE queries we need to make sure to add the extra information necessary to get hashed space partitions with the native postgres inheritance expansion working.
svenklemm
force-pushed
the
rowmark
branch
2 times, most recently
from
1850337
to
4b208fc
Compare
This patch adjusts the operator logic for valid space dimension constraints to no longer look for an exact match on both sides of the operator but instead allow mismatched datatypes. Previously a constraint like `col = value` would require `col` and `value` to have matching datatype with this change `col` and `value` can be different datatype as long as they have equality operator in btree family. Mismatching datatype can happen commonly when using int8 columns and comparing them with integer literals. Integer literals default to int4 so the datatypes would not match unless special care has been taken in writing the constraints and therefore the optimization would never apply in those cases.
Merged
jnidzwetzki
added a commit
to jnidzwetzki/timescaledb
that referenced
this pull request
Oct 5, 2022
This release is a patch release. We recommend that you upgrade at the next available opportunity. **Bugfixes** * timescale#4454 Keep locks after reading job status * timescale#4658 Fix error when querying a compressed hypertable with compress_segmentby on an enum column * timescale#4671 Fix a possible error while flushing the COPY data * timescale#4675 Fix bad TupleTableSlot drop * timescale#4676 Fix a deadlock when decompressing chunks and performing SELECTs * timescale#4685 Fix chunk exclusion for space partitions in SELECT FOR UPDATE queries * timescale#4694 Change parameter names of cagg_migrate procedure * timescale#4698 Do not use row-by-row fetcher for parameterized plans * timescale#4711 Remove support for procedures as custom checks * timescale#4712 Fix assertion failure in constify_now * timescale#4713 Fix Continuous Aggregate migration policies * timescale#4720 Fix chunk exclusion for prepared statements and dst changes * timescale#4726 Fix gapfill function signature * timescale#4737 Fix join on time column of compressed chunk * timescale#4738 Fix error when waiting for remote COPY to finish * timescale#4739 Fix continuous aggregate migrate check constraint * timescale#4760 Fix segfault when INNER JOINing hypertables * timescale#4767 Fix permission issues on index creation for CAggs **Thanks** * @boxhock and @cocowalla for reporting a segfault when JOINing hypertables * @carobme for reporting constraint error during continuous aggregate migration * @choisnetm, @dustinsorensen, @jayadevanm and @joeyberkovitz for reporting a problem with JOINs on compressed hypertables * @daniel-k for reporting a background worker crash * @justinpryzby for reporting an error when compressing very wide tables * @maxtwardowski for reporting problems with chunk exclusion and space partitions * @yuezhihan for reporting GROUP BY error when having compress_segmentby on an enum column
jnidzwetzki
added a commit
to jnidzwetzki/timescaledb
that referenced
this pull request
Oct 5, 2022
This release is a patch release. We recommend that you upgrade at the next available opportunity. **Bugfixes** * timescale#4454 Keep locks after reading job status * timescale#4658 Fix error when querying a compressed hypertable with compress_segmentby on an enum column * timescale#4671 Fix a possible error while flushing the COPY data * timescale#4675 Fix bad TupleTableSlot drop * timescale#4676 Fix a deadlock when decompressing chunks and performing SELECTs * timescale#4685 Fix chunk exclusion for space partitions in SELECT FOR UPDATE queries * timescale#4694 Change parameter names of cagg_migrate procedure * timescale#4698 Do not use row-by-row fetcher for parameterized plans * timescale#4711 Remove support for procedures as custom checks * timescale#4712 Fix assertion failure in constify_now * timescale#4713 Fix Continuous Aggregate migration policies * timescale#4720 Fix chunk exclusion for prepared statements and dst changes * timescale#4726 Fix gapfill function signature * timescale#4737 Fix join on time column of compressed chunk * timescale#4738 Fix error when waiting for remote COPY to finish * timescale#4739 Fix continuous aggregate migrate check constraint * timescale#4760 Fix segfault when INNER JOINing hypertables * timescale#4767 Fix permission issues on index creation for CAggs **Thanks** * @boxhock and @cocowalla for reporting a segfault when JOINing hypertables * @carobme for reporting constraint error during continuous aggregate migration * @choisnetm, @dustinsorensen, @jayadevanm and @joeyberkovitz for reporting a problem with JOINs on compressed hypertables * @daniel-k for reporting a background worker crash * @justinpryzby for reporting an error when compressing very wide tables * @maxtwardowski for reporting problems with chunk exclusion and space partitions * @yuezhihan for reporting GROUP BY error when having compress_segmentby on an enum column
jnidzwetzki
added a commit
to jnidzwetzki/timescaledb
that referenced
this pull request
Oct 5, 2022
This release is a patch release. We recommend that you upgrade at the next available opportunity. **Bugfixes** * timescale#4454 Keep locks after reading job status * timescale#4658 Fix error when querying a compressed hypertable with compress_segmentby on an enum column * timescale#4671 Fix a possible error while flushing the COPY data * timescale#4675 Fix bad TupleTableSlot drop * timescale#4676 Fix a deadlock when decompressing chunks and performing SELECTs * timescale#4685 Fix chunk exclusion for space partitions in SELECT FOR UPDATE queries * timescale#4694 Change parameter names of cagg_migrate procedure * timescale#4698 Do not use row-by-row fetcher for parameterized plans * timescale#4711 Remove support for procedures as custom checks * timescale#4712 Fix assertion failure in constify_now * timescale#4713 Fix Continuous Aggregate migration policies * timescale#4720 Fix chunk exclusion for prepared statements and dst changes * timescale#4726 Fix gapfill function signature * timescale#4737 Fix join on time column of compressed chunk * timescale#4738 Fix error when waiting for remote COPY to finish * timescale#4739 Fix continuous aggregate migrate check constraint * timescale#4760 Fix segfault when INNER JOINing hypertables * timescale#4767 Fix permission issues on index creation for CAggs **Thanks** * @boxhock and @cocowalla for reporting a segfault when JOINing hypertables * @carobme for reporting constraint error during continuous aggregate migration * @choisnetm, @dustinsorensen, @jayadevanm and @joeyberkovitz for reporting a problem with JOINs on compressed hypertables * @daniel-k for reporting a background worker crash * @justinpryzby for reporting an error when compressing very wide tables * @maxtwardowski for reporting problems with chunk exclusion and space partitions * @yuezhihan for reporting GROUP BY error when having compress_segmentby on an enum column
jnidzwetzki
added a commit
to jnidzwetzki/timescaledb
that referenced
this pull request
Oct 5, 2022
This release is a patch release. We recommend that you upgrade at the next available opportunity. **Bugfixes** * timescale#4454 Keep locks after reading job status * timescale#4658 Fix error when querying a compressed hypertable with compress_segmentby on an enum column * timescale#4671 Fix a possible error while flushing the COPY data * timescale#4675 Fix bad TupleTableSlot drop * timescale#4676 Fix a deadlock when decompressing chunks and performing SELECTs * timescale#4685 Fix chunk exclusion for space partitions in SELECT FOR UPDATE queries * timescale#4694 Change parameter names of cagg_migrate procedure * timescale#4698 Do not use row-by-row fetcher for parameterized plans * timescale#4711 Remove support for procedures as custom checks * timescale#4712 Fix assertion failure in constify_now * timescale#4713 Fix Continuous Aggregate migration policies * timescale#4720 Fix chunk exclusion for prepared statements and dst changes * timescale#4726 Fix gapfill function signature * timescale#4737 Fix join on time column of compressed chunk * timescale#4738 Fix error when waiting for remote COPY to finish * timescale#4739 Fix continuous aggregate migrate check constraint * timescale#4760 Fix segfault when INNER JOINing hypertables * timescale#4767 Fix permission issues on index creation for CAggs **Thanks** * @boxhock and @cocowalla for reporting a segfault when JOINing hypertables * @carobme for reporting constraint error during continuous aggregate migration * @choisnetm, @dustinsorensen, @jayadevanm and @joeyberkovitz for reporting a problem with JOINs on compressed hypertables * @daniel-k for reporting a background worker crash * @justinpryzby for reporting an error when compressing very wide tables * @maxtwardowski for reporting problems with chunk exclusion and space partitions * @yuezhihan for reporting GROUP BY error when having compress_segmentby on an enum column
jnidzwetzki
added a commit
to jnidzwetzki/timescaledb
that referenced
this pull request
Oct 5, 2022
This release is a patch release. We recommend that you upgrade at the next available opportunity. **Bugfixes** * timescale#4454 Keep locks after reading job status * timescale#4658 Fix error when querying a compressed hypertable with compress_segmentby on an enum column * timescale#4671 Fix a possible error while flushing the COPY data * timescale#4675 Fix bad TupleTableSlot drop * timescale#4676 Fix a deadlock when decompressing chunks and performing SELECTs * timescale#4685 Fix chunk exclusion for space partitions in SELECT FOR UPDATE queries * timescale#4694 Change parameter names of cagg_migrate procedure * timescale#4698 Do not use row-by-row fetcher for parameterized plans * timescale#4711 Remove support for procedures as custom checks * timescale#4712 Fix assertion failure in constify_now * timescale#4713 Fix Continuous Aggregate migration policies * timescale#4720 Fix chunk exclusion for prepared statements and dst changes * timescale#4726 Fix gapfill function signature * timescale#4737 Fix join on time column of compressed chunk * timescale#4738 Fix error when waiting for remote COPY to finish * timescale#4739 Fix continuous aggregate migrate check constraint * timescale#4760 Fix segfault when INNER JOINing hypertables * timescale#4767 Fix permission issues on index creation for CAggs **Thanks** * @boxhock and @cocowalla for reporting a segfault when JOINing hypertables * @carobme for reporting constraint error during continuous aggregate migration * @choisnetm, @dustinsorensen, @jayadevanm and @joeyberkovitz for reporting a problem with JOINs on compressed hypertables * @daniel-k for reporting a background worker crash * @justinpryzby for reporting an error when compressing very wide tables * @maxtwardowski for reporting problems with chunk exclusion and space partitions * @yuezhihan for reporting GROUP BY error when having compress_segmentby on an enum column
jnidzwetzki
added a commit
to jnidzwetzki/timescaledb
that referenced
this pull request
Oct 5, 2022
This release is a patch release. We recommend that you upgrade at the next available opportunity. **Bugfixes** * timescale#4454 Keep locks after reading job status * timescale#4658 Fix error when querying a compressed hypertable with compress_segmentby on an enum column * timescale#4671 Fix a possible error while flushing the COPY data * timescale#4675 Fix bad TupleTableSlot drop * timescale#4676 Fix a deadlock when decompressing chunks and performing SELECTs * timescale#4685 Fix chunk exclusion for space partitions in SELECT FOR UPDATE queries * timescale#4694 Change parameter names of cagg_migrate procedure * timescale#4698 Do not use row-by-row fetcher for parameterized plans * timescale#4711 Remove support for procedures as custom checks * timescale#4712 Fix assertion failure in constify_now * timescale#4713 Fix Continuous Aggregate migration policies * timescale#4720 Fix chunk exclusion for prepared statements and dst changes * timescale#4726 Fix gapfill function signature * timescale#4737 Fix join on time column of compressed chunk * timescale#4738 Fix error when waiting for remote COPY to finish * timescale#4739 Fix continuous aggregate migrate check constraint * timescale#4760 Fix segfault when INNER JOINing hypertables * timescale#4767 Fix permission issues on index creation for CAggs **Thanks** * @boxhock and @cocowalla for reporting a segfault when JOINing hypertables * @carobme for reporting constraint error during continuous aggregate migration * @choisnetm, @dustinsorensen, @jayadevanm and @joeyberkovitz for reporting a problem with JOINs on compressed hypertables * @daniel-k for reporting a background worker crash * @justinpryzby for reporting an error when compressing very wide tables * @maxtwardowski for reporting problems with chunk exclusion and space partitions * @yuezhihan for reporting GROUP BY error when having compress_segmentby on an enum column
jnidzwetzki
added a commit
to jnidzwetzki/timescaledb
that referenced
this pull request
Oct 5, 2022
This release is a patch release. We recommend that you upgrade at the next available opportunity. **Bugfixes** * timescale#4454 Keep locks after reading job status * timescale#4658 Fix error when querying a compressed hypertable with compress_segmentby on an enum column * timescale#4671 Fix a possible error while flushing the COPY data * timescale#4675 Fix bad TupleTableSlot drop * timescale#4676 Fix a deadlock when decompressing chunks and performing SELECTs * timescale#4685 Fix chunk exclusion for space partitions in SELECT FOR UPDATE queries * timescale#4694 Change parameter names of cagg_migrate procedure * timescale#4698 Do not use row-by-row fetcher for parameterized plans * timescale#4711 Remove support for procedures as custom checks * timescale#4712 Fix assertion failure in constify_now * timescale#4713 Fix Continuous Aggregate migration policies * timescale#4720 Fix chunk exclusion for prepared statements and dst changes * timescale#4726 Fix gapfill function signature * timescale#4737 Fix join on time column of compressed chunk * timescale#4738 Fix error when waiting for remote COPY to finish * timescale#4739 Fix continuous aggregate migrate check constraint * timescale#4760 Fix segfault when INNER JOINing hypertables * timescale#4767 Fix permission issues on index creation for CAggs **Thanks** * @boxhock and @cocowalla for reporting a segfault when JOINing hypertables * @carobme for reporting constraint error during continuous aggregate migration * @choisnetm, @dustinsorensen, @jayadevanm and @joeyberkovitz for reporting a problem with JOINs on compressed hypertables * @daniel-k for reporting a background worker crash * @justinpryzby for reporting an error when compressing very wide tables * @maxtwardowski for reporting problems with chunk exclusion and space partitions * @yuezhihan for reporting GROUP BY error when having compress_segmentby on an enum column
jnidzwetzki
added a commit
to jnidzwetzki/timescaledb
that referenced
this pull request
Oct 5, 2022
This release is a patch release. We recommend that you upgrade at the next available opportunity. **Bugfixes** * timescale#4454 Keep locks after reading job status * timescale#4658 Fix error when querying a compressed hypertable with compress_segmentby on an enum column * timescale#4671 Fix a possible error while flushing the COPY data * timescale#4675 Fix bad TupleTableSlot drop * timescale#4676 Fix a deadlock when decompressing chunks and performing SELECTs * timescale#4685 Fix chunk exclusion for space partitions in SELECT FOR UPDATE queries * timescale#4694 Change parameter names of cagg_migrate procedure * timescale#4698 Do not use row-by-row fetcher for parameterized plans * timescale#4711 Remove support for procedures as custom checks * timescale#4712 Fix assertion failure in constify_now * timescale#4713 Fix Continuous Aggregate migration policies * timescale#4720 Fix chunk exclusion for prepared statements and dst changes * timescale#4726 Fix gapfill function signature * timescale#4737 Fix join on time column of compressed chunk * timescale#4738 Fix error when waiting for remote COPY to finish * timescale#4739 Fix continuous aggregate migrate check constraint * timescale#4760 Fix segfault when INNER JOINing hypertables * timescale#4767 Fix permission issues on index creation for CAggs **Thanks** * @boxhock and @cocowalla for reporting a segfault when JOINing hypertables * @carobme for reporting constraint error during continuous aggregate migration * @choisnetm, @dustinsorensen, @jayadevanm and @joeyberkovitz for reporting a problem with JOINs on compressed hypertables * @daniel-k for reporting a background worker crash * @justinpryzby for reporting an error when compressing very wide tables * @maxtwardowski for reporting problems with chunk exclusion and space partitions * @yuezhihan for reporting GROUP BY error when having compress_segmentby on an enum column
jnidzwetzki
added a commit
that referenced
this pull request
Oct 5, 2022
This release is a patch release. We recommend that you upgrade at the next available opportunity. **Bugfixes** * #4454 Keep locks after reading job status * #4658 Fix error when querying a compressed hypertable with compress_segmentby on an enum column * #4671 Fix a possible error while flushing the COPY data * #4675 Fix bad TupleTableSlot drop * #4676 Fix a deadlock when decompressing chunks and performing SELECTs * #4685 Fix chunk exclusion for space partitions in SELECT FOR UPDATE queries * #4694 Change parameter names of cagg_migrate procedure * #4698 Do not use row-by-row fetcher for parameterized plans * #4711 Remove support for procedures as custom checks * #4712 Fix assertion failure in constify_now * #4713 Fix Continuous Aggregate migration policies * #4720 Fix chunk exclusion for prepared statements and dst changes * #4726 Fix gapfill function signature * #4737 Fix join on time column of compressed chunk * #4738 Fix error when waiting for remote COPY to finish * #4739 Fix continuous aggregate migrate check constraint * #4760 Fix segfault when INNER JOINing hypertables * #4767 Fix permission issues on index creation for CAggs **Thanks** * @boxhock and @cocowalla for reporting a segfault when JOINing hypertables * @carobme for reporting constraint error during continuous aggregate migration * @choisnetm, @dustinsorensen, @jayadevanm and @joeyberkovitz for reporting a problem with JOINs on compressed hypertables * @daniel-k for reporting a background worker crash * @justinpryzby for reporting an error when compressing very wide tables * @maxtwardowski for reporting problems with chunk exclusion and space partitions * @yuezhihan for reporting GROUP BY error when having compress_segmentby on an enum column
jnidzwetzki
added a commit
that referenced
this pull request
Oct 6, 2022
This release is a patch release. We recommend that you upgrade at the next available opportunity. **Bugfixes** * #4454 Keep locks after reading job status * #4658 Fix error when querying a compressed hypertable with compress_segmentby on an enum column * #4671 Fix a possible error while flushing the COPY data * #4675 Fix bad TupleTableSlot drop * #4676 Fix a deadlock when decompressing chunks and performing SELECTs * #4685 Fix chunk exclusion for space partitions in SELECT FOR UPDATE queries * #4694 Change parameter names of cagg_migrate procedure * #4698 Do not use row-by-row fetcher for parameterized plans * #4711 Remove support for procedures as custom checks * #4712 Fix assertion failure in constify_now * #4713 Fix Continuous Aggregate migration policies * #4720 Fix chunk exclusion for prepared statements and dst changes * #4726 Fix gapfill function signature * #4737 Fix join on time column of compressed chunk * #4738 Fix error when waiting for remote COPY to finish * #4739 Fix continuous aggregate migrate check constraint * #4760 Fix segfault when INNER JOINing hypertables * #4767 Fix permission issues on index creation for CAggs **Thanks** * @boxhock and @cocowalla for reporting a segfault when JOINing hypertables * @carobme for reporting constraint error during continuous aggregate migration * @choisnetm, @dustinsorensen, @jayadevanm and @joeyberkovitz for reporting a problem with JOINs on compressed hypertables * @daniel-k for reporting a background worker crash * @justinpryzby for reporting an error when compressing very wide tables * @maxtwardowski for reporting problems with chunk exclusion and space partitions * @yuezhihan for reporting GROUP BY error when having compress_segmentby on an enum column
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Since we do not use our own hypertable expansion for SELECT FOR UPDATE
queries we need to make sure to add the extra information necessary to
get hashed space partitions with the native postgres inheritance
expansion working.
Fixes #4683
Disable-check: commit-count