-
Notifications
You must be signed in to change notification settings - Fork 854
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix SkipScan path generation with constant DISTINCT column #3661
Conversation
a30cc03
to
d4a99d9
Compare
Just need to add |
Codecov Report
@@ Coverage Diff @@
## master #3661 +/- ##
=======================================
Coverage 90.79% 90.79%
=======================================
Files 212 212
Lines 36548 36549 +1
=======================================
+ Hits 33182 33185 +3
+ Misses 3366 3364 -2
Continue to review full report at Codecov.
|
075da3c
to
43aebc3
Compare
@@ -3909,3 +3909,35 @@ EXPLAIN (costs off, timing off, summary off) SELECT DISTINCT ON (dev_name) dev_n | |||
Index Cond: (dev_name > NULL::text) | |||
(15 rows) | |||
|
|||
-- #3629 skipscan with constant skipscan column in where clause |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This comment is a bit misleading. We don't have an index with "a". So cannot apply skipscan?
tsl/src/nodes/skip_scan/planner.c
Outdated
@@ -536,7 +536,21 @@ build_skip_qual(PlannerInfo *root, SkipScanPath *skip_scan_path, IndexPath *inde | |||
Oid column_type = exprType((Node *) var); | |||
Oid column_collation = get_typcollation(column_type); | |||
TypeCacheEntry *tce = lookup_type_cache(column_type, 0); | |||
|
|||
/* | |||
* If we have an order by clause with columns that have constraints in the WHERE clause |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
* If we have an order by clause with columns that have constraints in the WHERE clause | |
Skipscan is not applicable for the following case: | |
We might have a path with an index that produces the correct pathkeys for the target ordering without actually including all the columns of the ORDER BY. If the path uses an index that does not include the distinct column, we cannot use it for skipscan and have to discard this path from skipscan generation. | |
This happens, for instance, when we have an order by clause ( like ORDER BY a, b) with constraints in the WHERE clause (like WHERE a = <constant>) . "a" can now be removed from the Pathkeys (since it is a constant) and the query can be satisfied by using an index on just column "b". |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes look good. Minor comment edits requested.
When a DISTINCT query has a WHERE clause that constifies the DISTINCT column the query might use an index that does not have include the DISTINCT column even though it is referenced in the ORDER BY clause. The skipscan path generation would error on any path with such a configuration. This patch changes the path generation code to skip generating SkipScan path under these circumstances. Fixes timescale#3629
This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading This release adds several and long-awaited/wanted features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Major Features** * timescale#3435 Add continuous aggregates for distributed hypertables * timescale#3034 Add support for PostgreSQL 14 * timescale#3505 Add support for timezones in `time_bucket_ng()` **Minor Features** * timescale#3598 Improve evaluation of stable functions such as now() on access node **Bugfixes** * timescale#3580 Fix memory context bug executing TRUNCATE * timescale#3654 Fix index attnum mapping in reorder_chunk * timescale#3661 Fix SkipScan path generation with constant DISTINCT column * timescale#3708 Fix crash in get_aggsplit * timescale#3709 Fix ordered append pathkey check * timescale#3728 Fix SkipScan with varchar column **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() Disable-check: commit-count
This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading This release adds several and long-awaited/wanted features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Major Features** * timescale#3435 Add continuous aggregates for distributed hypertables * timescale#3034 Add support for PostgreSQL 14 * timescale#3505 Add support for timezones in `time_bucket_ng()` **Minor Features** * timescale#3598 Improve evaluation of stable functions such as now() on access node **Bugfixes** * timescale#3580 Fix memory context bug executing TRUNCATE * timescale#3654 Fix index attnum mapping in reorder_chunk * timescale#3661 Fix SkipScan path generation with constant DISTINCT column * timescale#3708 Fix crash in get_aggsplit * timescale#3709 Fix ordered append pathkey check * timescale#3728 Fix SkipScan with varchar column **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() Disable-check: commit-count
This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading This release adds several and long-awaited/wanted features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Major Features** * timescale#3435 Add continuous aggregates for distributed hypertables * timescale#3034 Add support for PostgreSQL 14 * timescale#3505 Add support for timezones in `time_bucket_ng()` **Minor Features** * timescale#3598 Improve evaluation of stable functions such as now() on access node **Bugfixes** * timescale#3580 Fix memory context bug executing TRUNCATE * timescale#3654 Fix index attnum mapping in reorder_chunk * timescale#3661 Fix SkipScan path generation with constant DISTINCT column * timescale#3708 Fix crash in get_aggsplit * timescale#3709 Fix ordered append pathkey check * timescale#3728 Fix SkipScan with varchar column **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() Disable-check: commit-count
This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading This release adds several and long-awaited/wanted features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Major Features** * timescale#3435 Add continuous aggregates for distributed hypertables * timescale#3034 Add support for PostgreSQL 14 * timescale#3505 Add support for timezones in `time_bucket_ng()` **Minor Features** * timescale#3598 Improve evaluation of stable functions such as now() on access node **Bugfixes** * timescale#3580 Fix memory context bug executing TRUNCATE * timescale#3654 Fix index attnum mapping in reorder_chunk * timescale#3661 Fix SkipScan path generation with constant DISTINCT column * timescale#3708 Fix crash in get_aggsplit * timescale#3709 Fix ordered append pathkey check * timescale#3728 Fix SkipScan with varchar column **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() Disable-check: commit-count
This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading This release adds several and long-awaited/wanted features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Major Features** * timescale#3435 Add continuous aggregates for distributed hypertables * timescale#3034 Add support for PostgreSQL 14 * timescale#3505 Add support for timezones in `time_bucket_ng()` **Minor Features** * timescale#3598 Improve evaluation of stable functions such as now() on access node **Bugfixes** * timescale#3580 Fix memory context bug executing TRUNCATE * timescale#3654 Fix index attnum mapping in reorder_chunk * timescale#3661 Fix SkipScan path generation with constant DISTINCT column * timescale#3708 Fix crash in get_aggsplit * timescale#3709 Fix ordered append pathkey check * timescale#3728 Fix SkipScan with varchar column **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() Disable-check: commit-count
This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Features** * timescale#3034 Add support for PostgreSQL 14 * timescale#3435 Add continuous aggregates for distributed hypertables * timescale#3505 Add support for timezones in `time_bucket_ng()` * timescale#3598 Improve evaluation of stable functions such as now() on access node * timescale#3717 Support transparent decompression on individual chunks **Bugfixes** * timescale#3580 Fix memory context bug executing TRUNCATE * timescale#3592 Allow alter column type on distributed hypertable * timescale#3618 Fix execution of refresh_caggs from user actions * timescale#3625 Add shared dependencies when creating chunk * timescale#3626 Fix memory context bug executing TRUNCATE * timescale#3627 Schema qualify UDTs in multi-node * timescale#3638 Allow owner change of a data node * timescale#3654 Fix index attnum mapping in reorder_chunk * timescale#3661 Fix SkipScan path generation with constant DISTINCT column * timescale#3667 Fix compress_policy for multi txn handling * timescale#3673 Fix distributed hypertable DROP within a procedure * timescale#3701 Allow anyone to use size utilities on distributed hypertables * timescale#3708 Fix crash in get_aggsplit * timescale#3709 Fix ordered append pathkey check * timescale#3712 Fix GRANT/REVOKE ALL IN SCHEMA handling * timescale#3724 Fix inserts into compressed chunks on hypertables with caggs * timescale#3727 Fix DirectFunctionCall crash in distributed_exec * timescale#3728 Fix SkipScan with varchar column * timescale#3733 Fix ANALYZE crash with custom statistics for custom types * timescale#3747 Always reset expr context in DecompressChunk **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() * @abolognino for reporting an issue with INSERTs into compressed hypertables that have cagg * @tanglebones for reporting the ANALYZE crash with custom types on multinode
This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Features** * timescale#3034 Add support for PostgreSQL 14 * timescale#3435 Add continuous aggregates for distributed hypertables * timescale#3505 Add support for timezones in `time_bucket_ng()` **Bugfixes** * timescale#3580 Fix memory context bug executing TRUNCATE * timescale#3592 Allow alter column type on distributed hypertable * timescale#3598 Improve evaluation of stable functions such as now() on access node * timescale#3618 Fix execution of refresh_caggs from user actions * timescale#3625 Add shared dependencies when creating chunk * timescale#3626 Fix memory context bug executing TRUNCATE * timescale#3627 Schema qualify UDTs in multi-node * timescale#3638 Allow owner change of a data node * timescale#3654 Fix index attnum mapping in reorder_chunk * timescale#3661 Fix SkipScan path generation with constant DISTINCT column * timescale#3667 Fix compress_policy for multi txn handling * timescale#3673 Fix distributed hypertable DROP within a procedure * timescale#3701 Allow anyone to use size utilities on distributed hypertables * timescale#3708 Fix crash in get_aggsplit * timescale#3709 Fix ordered append pathkey check * timescale#3712 Fix GRANT/REVOKE ALL IN SCHEMA handling * timescale#3717 Support transparent decompression on individual chunks * timescale#3724 Fix inserts into compressed chunks on hypertables with caggs * timescale#3727 Fix DirectFunctionCall crash in distributed_exec * timescale#3728 Fix SkipScan with varchar column * timescale#3733 Fix ANALYZE crash with custom statistics for custom types * timescale#3747 Always reset expr context in DecompressChunk **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() * @abolognino for reporting an issue with INSERTs into compressed hypertables that have cagg * @tanglebones for reporting the ANALYZE crash with custom types on multinode
This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Features** * #3034 Add support for PostgreSQL 14 * #3435 Add continuous aggregates for distributed hypertables * #3505 Add support for timezones in `time_bucket_ng()` **Bugfixes** * #3580 Fix memory context bug executing TRUNCATE * #3592 Allow alter column type on distributed hypertable * #3598 Improve evaluation of stable functions such as now() on access node * #3618 Fix execution of refresh_caggs from user actions * #3625 Add shared dependencies when creating chunk * #3626 Fix memory context bug executing TRUNCATE * #3627 Schema qualify UDTs in multi-node * #3638 Allow owner change of a data node * #3654 Fix index attnum mapping in reorder_chunk * #3661 Fix SkipScan path generation with constant DISTINCT column * #3667 Fix compress_policy for multi txn handling * #3673 Fix distributed hypertable DROP within a procedure * #3701 Allow anyone to use size utilities on distributed hypertables * #3708 Fix crash in get_aggsplit * #3709 Fix ordered append pathkey check * #3712 Fix GRANT/REVOKE ALL IN SCHEMA handling * #3717 Support transparent decompression on individual chunks * #3724 Fix inserts into compressed chunks on hypertables with caggs * #3727 Fix DirectFunctionCall crash in distributed_exec * #3728 Fix SkipScan with varchar column * #3733 Fix ANALYZE crash with custom statistics for custom types * #3747 Always reset expr context in DecompressChunk **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() * @abolognino for reporting an issue with INSERTs into compressed hypertables that have cagg * @tanglebones for reporting the ANALYZE crash with custom types on multinode
This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Features** * #3034 Add support for PostgreSQL 14 * #3435 Add continuous aggregates for distributed hypertables * #3505 Add support for timezones in `time_bucket_ng()` **Bugfixes** * #3580 Fix memory context bug executing TRUNCATE * #3592 Allow alter column type on distributed hypertable * #3598 Improve evaluation of stable functions such as now() on access node * #3618 Fix execution of refresh_caggs from user actions * #3625 Add shared dependencies when creating chunk * #3626 Fix memory context bug executing TRUNCATE * #3627 Schema qualify UDTs in multi-node * #3638 Allow owner change of a data node * #3654 Fix index attnum mapping in reorder_chunk * #3661 Fix SkipScan path generation with constant DISTINCT column * #3667 Fix compress_policy for multi txn handling * #3673 Fix distributed hypertable DROP within a procedure * #3701 Allow anyone to use size utilities on distributed hypertables * #3708 Fix crash in get_aggsplit * #3709 Fix ordered append pathkey check * #3712 Fix GRANT/REVOKE ALL IN SCHEMA handling * #3717 Support transparent decompression on individual chunks * #3724 Fix inserts into compressed chunks on hypertables with caggs * #3727 Fix DirectFunctionCall crash in distributed_exec * #3728 Fix SkipScan with varchar column * #3733 Fix ANALYZE crash with custom statistics for custom types * #3747 Always reset expr context in DecompressChunk **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() * @abolognino for reporting an issue with INSERTs into compressed hypertables that have cagg * @tanglebones for reporting the ANALYZE crash with custom types on multinode
This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Features** * #3034 Add support for PostgreSQL 14 * #3435 Add continuous aggregates for distributed hypertables * #3505 Add support for timezones in `time_bucket_ng()` **Bugfixes** * #3580 Fix memory context bug executing TRUNCATE * #3592 Allow alter column type on distributed hypertable * #3598 Improve evaluation of stable functions such as now() on access node * #3618 Fix execution of refresh_caggs from user actions * #3625 Add shared dependencies when creating chunk * #3626 Fix memory context bug executing TRUNCATE * #3627 Schema qualify UDTs in multi-node * #3638 Allow owner change of a data node * #3654 Fix index attnum mapping in reorder_chunk * #3661 Fix SkipScan path generation with constant DISTINCT column * #3667 Fix compress_policy for multi txn handling * #3673 Fix distributed hypertable DROP within a procedure * #3701 Allow anyone to use size utilities on distributed hypertables * #3708 Fix crash in get_aggsplit * #3709 Fix ordered append pathkey check * #3712 Fix GRANT/REVOKE ALL IN SCHEMA handling * #3717 Support transparent decompression on individual chunks * #3724 Fix inserts into compressed chunks on hypertables with caggs * #3727 Fix DirectFunctionCall crash in distributed_exec * #3728 Fix SkipScan with varchar column * #3733 Fix ANALYZE crash with custom statistics for custom types * #3747 Always reset expr context in DecompressChunk **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() * @abolognino for reporting an issue with INSERTs into compressed hypertables that have cagg * @tanglebones for reporting the ANALYZE crash with custom types on multinode
This release adds major new features since the 2.4.2 release. We deem it moderate priority for upgrading. This release includes these noteworthy features: * Continuous Aggregates for Distributed Hypertables * Support for PostgreSQL 14 * Experimental: Support for timezones in `time_bucket_ng()`, including the `origin` argument This release also includes several bug fixes. **Features** * #3034 Add support for PostgreSQL 14 * #3435 Add continuous aggregates for distributed hypertables * #3505 Add support for timezones in `time_bucket_ng()` **Bugfixes** * #3580 Fix memory context bug executing TRUNCATE * #3592 Allow alter column type on distributed hypertable * #3598 Improve evaluation of stable functions such as now() on access node * #3618 Fix execution of refresh_caggs from user actions * #3625 Add shared dependencies when creating chunk * #3626 Fix memory context bug executing TRUNCATE * #3627 Schema qualify UDTs in multi-node * #3638 Allow owner change of a data node * #3654 Fix index attnum mapping in reorder_chunk * #3661 Fix SkipScan path generation with constant DISTINCT column * #3667 Fix compress_policy for multi txn handling * #3673 Fix distributed hypertable DROP within a procedure * #3701 Allow anyone to use size utilities on distributed hypertables * #3708 Fix crash in get_aggsplit * #3709 Fix ordered append pathkey check * #3712 Fix GRANT/REVOKE ALL IN SCHEMA handling * #3717 Support transparent decompression on individual chunks * #3724 Fix inserts into compressed chunks on hypertables with caggs * #3727 Fix DirectFunctionCall crash in distributed_exec * #3728 Fix SkipScan with varchar column * #3733 Fix ANALYZE crash with custom statistics for custom types * #3747 Always reset expr context in DecompressChunk **Thanks** * @binakot and @sebvett for reporting an issue with DISTINCT queries * @hardikm10, @DavidPavlicek and @pafiti for reporting bugs on TRUNCATE * @mjf for reporting an issue with ordered append and JOINs * @phemmer for reporting the issues on multinode with aggregate queries and evaluation of now() * @abolognino for reporting an issue with INSERTs into compressed hypertables that have cagg * @tanglebones for reporting the ANALYZE crash with custom types on multinode
When a DISTINCT query has a WHERE clause that constifies the
DISTINCT column the query might use an index that does not have
include the DISTINCT column even though it is referenced in the
ORDER BY clause. The skipscan path generation would error on any
path with such a configuration. This patch changes the path
generation code to skip generating SkipScan path under these
circumstances.
Fixes #3629