Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix const null clauses in runtime chunk exclusion #4880

Merged
merged 2 commits into from Nov 15, 2022
Merged

Fix const null clauses in runtime chunk exclusion #4880

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9f9751e
Fix const null clauses in runtime chunk exclusion
akuzm Nov 15, 2022
28a4310
Escape the quotes in gdb command
akuzm Nov 15, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/linux-build-and-test.yaml
View file
Open in desktop
Expand Up @@ -196,9 +196,9 @@ jobs:
sudo coredumpctl gdb <<<"
set verbose on
show debug-file-directory
printf "query = '%s'\n\n", debug_query_string
printf "'"'"query = '%s'\n\n"'"'", debug_query_string
frame function ExceptionalCondition
printf "condition = '%s'\n", conditionName
printf "'"'"condition = '%s'\n"'"'", conditionName
bt full
" | tee stacktrace.log
./scripts/bundle_coredumps.sh
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/sanitizer-build-and-test.yaml
View file
Open in desktop
Expand Up @@ -163,9 +163,9 @@ jobs:
sudo coredumpctl gdb <<<"
set verbose on
show debug-file-directory
printf "query = '%s'\n\n", debug_query_string
printf "'"'"query = '%s'\n\n"'"'", debug_query_string
frame function ExceptionalCondition
printf "condition = '%s'\n", conditionName
printf "'"'"condition = '%s'\n"'"'", conditionName
bt full
" | tee stacktrace.log
./scripts/bundle_coredumps.sh
Expand Down
22 changes: 13 additions & 9 deletions src/nodes/chunk_append/exec.c
View file
Open in desktop
Expand Up @@ -954,17 +954,21 @@ static bool
can_exclude_chunk(List *constraints, List *baserestrictinfo)
{
/*
* Regardless of the setting of constraint_exclusion, detect
* constant-FALSE-or-NULL restriction clauses. Because const-folding will
* reduce "anything AND FALSE" to just "FALSE", any such case should
* result in exactly one baserestrictinfo entry. This doesn't fire very
* often, but it seems cheap enough to be worth doing anyway. (Without
* this, we'd miss some optimizations that 9.5 and earlier found via much
* more roundabout methods.)
* Detect constant-FALSE-or-NULL restriction clauses. If we have such a
* clause, no rows from the chunk are going to match. Unlike the postgres
* analog of this code in relation_excluded_by_constraints, we can't expect
* a single const false restrictinfo in this case, because we don't try to
* fold the restrictinfos after evaluating the mutable functions.
* We have to check this separately from the subsequent predicate_refuted_by.
* That function can also work with the normal CHECK constraints, and they
* don't fail if the constraint evaluates to null given the restriction info.
* That's why it has to prove that the CHECK constraint evaluates to false,
* and this doesn't follow from having a const null restrictinfo.
*/
if (list_length(baserestrictinfo) == 1)
ListCell *lc;
foreach (lc, baserestrictinfo)
{
RestrictInfo *rinfo = (RestrictInfo *) linitial(baserestrictinfo);
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
Expr *clause = rinfo->clause;

if (clause && IsA(clause, Const) &&
Expand Down
126 changes: 126 additions & 0 deletions test/expected/null_exclusion.out
View file
Open in desktop
@@ -0,0 +1,126 @@
-- This file and its contents are licensed under the Apache License 2.0.
-- Please see the included NOTICE for copyright information and
-- LICENSE-APACHE for a copy of the license.
create table metrics(ts timestamp, id int, value float);
select create_hypertable('metrics', 'ts');
WARNING: column type "timestamp without time zone" used for "ts" does not follow best practices
NOTICE: adding not-null constraint to column "ts"
create_hypertable
----------------------
(1,public,metrics,t)
(1 row)

insert into metrics values ('2022-02-02 02:02:02', 2, 2.),
('2023-03-03 03:03:03', 3, 3.);
analyze metrics;
-- non-const condition
explain (analyze, costs off, summary off, timing off)
select * from metrics
where ts >= (select max(ts) from metrics);
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------
Custom Scan (ChunkAppend) on metrics (actual rows=1 loops=1)
Chunks excluded during runtime: 1
InitPlan 2 (returns $1)
-> Result (actual rows=1 loops=1)
InitPlan 1 (returns $0)
-> Limit (actual rows=1 loops=1)
-> Custom Scan (ChunkAppend) on metrics metrics_1 (actual rows=1 loops=1)
Order: metrics_1.ts DESC
-> Index Only Scan using _hyper_1_2_chunk_metrics_ts_idx on _hyper_1_2_chunk _hyper_1_2_chunk_1 (actual rows=1 loops=1)
Index Cond: (ts IS NOT NULL)
Heap Fetches: 1
-> Index Only Scan using _hyper_1_1_chunk_metrics_ts_idx on _hyper_1_1_chunk _hyper_1_1_chunk_1 (never executed)
Index Cond: (ts IS NOT NULL)
Heap Fetches: 0
-> Seq Scan on _hyper_1_1_chunk (never executed)
Filter: (ts >= $1)
-> Seq Scan on _hyper_1_2_chunk (actual rows=1 loops=1)
Filter: (ts >= $1)
(18 rows)

-- two non-const conditions
explain (analyze, costs off, summary off, timing off)
select * from metrics
where ts >= (select max(ts) from metrics)
and id = 1;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------
Custom Scan (ChunkAppend) on metrics (actual rows=0 loops=1)
Chunks excluded during runtime: 1
InitPlan 2 (returns $1)
-> Result (actual rows=1 loops=1)
InitPlan 1 (returns $0)
-> Limit (actual rows=1 loops=1)
-> Custom Scan (ChunkAppend) on metrics metrics_1 (actual rows=1 loops=1)
Order: metrics_1.ts DESC
-> Index Only Scan using _hyper_1_2_chunk_metrics_ts_idx on _hyper_1_2_chunk _hyper_1_2_chunk_1 (actual rows=1 loops=1)
Index Cond: (ts IS NOT NULL)
Heap Fetches: 1
-> Index Only Scan using _hyper_1_1_chunk_metrics_ts_idx on _hyper_1_1_chunk _hyper_1_1_chunk_1 (never executed)
Index Cond: (ts IS NOT NULL)
Heap Fetches: 0
-> Seq Scan on _hyper_1_1_chunk (never executed)
Filter: ((ts >= $1) AND (id = 1))
-> Seq Scan on _hyper_1_2_chunk (actual rows=0 loops=1)
Filter: ((ts >= $1) AND (id = 1))
Rows Removed by Filter: 1
(19 rows)

-- condition that becomes const null after evaluating the param
explain (analyze, costs off, summary off, timing off)
select * from metrics
where ts >= (select max(ts) from metrics where id = -1);
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------
Custom Scan (ChunkAppend) on metrics (actual rows=0 loops=1)
Chunks excluded during runtime: 2
InitPlan 2 (returns $1)
-> Result (actual rows=1 loops=1)
InitPlan 1 (returns $0)
-> Limit (actual rows=0 loops=1)
-> Custom Scan (ChunkAppend) on metrics metrics_1 (actual rows=0 loops=1)
Order: metrics_1.ts DESC
-> Index Scan using _hyper_1_2_chunk_metrics_ts_idx on _hyper_1_2_chunk _hyper_1_2_chunk_1 (actual rows=0 loops=1)
Index Cond: (ts IS NOT NULL)
Filter: (id = '-1'::integer)
Rows Removed by Filter: 1
-> Index Scan using _hyper_1_1_chunk_metrics_ts_idx on _hyper_1_1_chunk _hyper_1_1_chunk_1 (actual rows=0 loops=1)
Index Cond: (ts IS NOT NULL)
Filter: (id = '-1'::integer)
Rows Removed by Filter: 1
-> Seq Scan on _hyper_1_1_chunk (never executed)
Filter: (ts >= $1)
-> Seq Scan on _hyper_1_2_chunk (never executed)
Filter: (ts >= $1)
(20 rows)

-- const null condition and some other condition
explain (analyze, costs off, summary off, timing off)
select * from metrics
where ts >= (select max(ts) from metrics where id = -1)
and id = 1;
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------
Custom Scan (ChunkAppend) on metrics (actual rows=0 loops=1)
Chunks excluded during runtime: 2
InitPlan 2 (returns $1)
-> Result (actual rows=1 loops=1)
InitPlan 1 (returns $0)
-> Limit (actual rows=0 loops=1)
-> Custom Scan (ChunkAppend) on metrics metrics_1 (actual rows=0 loops=1)
Order: metrics_1.ts DESC
-> Index Scan using _hyper_1_2_chunk_metrics_ts_idx on _hyper_1_2_chunk _hyper_1_2_chunk_1 (actual rows=0 loops=1)
Index Cond: (ts IS NOT NULL)
Filter: (id = '-1'::integer)
Rows Removed by Filter: 1
-> Index Scan using _hyper_1_1_chunk_metrics_ts_idx on _hyper_1_1_chunk _hyper_1_1_chunk_1 (actual rows=0 loops=1)
Index Cond: (ts IS NOT NULL)
Filter: (id = '-1'::integer)
Rows Removed by Filter: 1
-> Seq Scan on _hyper_1_1_chunk (never executed)
Filter: ((ts >= $1) AND (id = 1))
-> Seq Scan on _hyper_1_2_chunk (never executed)
Filter: ((ts >= $1) AND (id = 1))
(20 rows)

1 change: 1 addition & 0 deletions test/sql/CMakeLists.txt
View file
Open in desktop
Expand Up @@ -30,6 +30,7 @@ set(TEST_FILES
insert_returning.sql
lateral.sql
misc.sql
null_exclusion.sql
partition.sql
partitioning.sql
pg_dump_unprivileged.sql
Expand Down
31 changes: 31 additions & 0 deletions test/sql/null_exclusion.sql
View file
Open in desktop
@@ -0,0 +1,31 @@
-- This file and its contents are licensed under the Apache License 2.0.
-- Please see the included NOTICE for copyright information and
-- LICENSE-APACHE for a copy of the license.

create table metrics(ts timestamp, id int, value float);
select create_hypertable('metrics', 'ts');
insert into metrics values ('2022-02-02 02:02:02', 2, 2.),
('2023-03-03 03:03:03', 3, 3.);
analyze metrics;

-- non-const condition
explain (analyze, costs off, summary off, timing off)
select * from metrics
where ts >= (select max(ts) from metrics);

-- two non-const conditions
explain (analyze, costs off, summary off, timing off)
select * from metrics
where ts >= (select max(ts) from metrics)
and id = 1;

-- condition that becomes const null after evaluating the param
explain (analyze, costs off, summary off, timing off)
select * from metrics
where ts >= (select max(ts) from metrics where id = -1);

-- const null condition and some other condition
explain (analyze, costs off, summary off, timing off)
select * from metrics
where ts >= (select max(ts) from metrics where id = -1)
and id = 1;
50 changes: 30 additions & 20 deletions tsl/test/expected/transparent_decompression_ordered_index-12.out
View file
Open in desktop
Expand Up @@ -95,6 +95,16 @@ ORDER BY c.id;
\set ECHO none
-- diff compressed and uncompressed results
:DIFF_CMD
-- This is to illustrate that we have some null device_id values. This fact
-- might influence the runtime chunk exclusion when doing joins on device_id.
select count(*) from metrics_ordered_idx
where extract(minute from time) = 0 and device_id is null
;
count
-------
1
(1 row)

\set PREFIX 'EXPLAIN (analyze, costs off, timing off, summary off)'
\set PREFIX_VERBOSE 'EXPLAIN (analyze, costs off, timing off, summary off, verbose)'
-- we disable parallelism here otherwise EXPLAIN ANALYZE output
Expand Down Expand Up @@ -328,20 +338,20 @@ WHERE extract(minute FROM d.time) = 0;
-> Custom Scan (ChunkAppend) on metrics_ordered_idx m_1 (actual rows=0 loops=389)
Order: m_1."time" DESC
Hypertables excluded during runtime: 0
-> Custom Scan (DecompressChunk) on _hyper_1_5_chunk m_2 (actual rows=0 loops=389)
-> Index Scan Backward using compress_hyper_2_10_chunk__compressed_hypertable_2_device_id_de on compress_hyper_2_10_chunk compress_hyper_2_10_chunk_1 (actual rows=0 loops=389)
-> Custom Scan (DecompressChunk) on _hyper_1_5_chunk m_2 (actual rows=0 loops=388)
-> Index Scan Backward using compress_hyper_2_10_chunk__compressed_hypertable_2_device_id_de on compress_hyper_2_10_chunk compress_hyper_2_10_chunk_1 (actual rows=0 loops=388)
Index Cond: ((device_id = d.device_id) AND (device_id_peer = 3))
-> Custom Scan (DecompressChunk) on _hyper_1_4_chunk m_3 (actual rows=0 loops=389)
-> Index Scan Backward using compress_hyper_2_9_chunk__compressed_hypertable_2_device_id_dev on compress_hyper_2_9_chunk compress_hyper_2_9_chunk_1 (actual rows=0 loops=389)
-> Custom Scan (DecompressChunk) on _hyper_1_4_chunk m_3 (actual rows=0 loops=388)
-> Index Scan Backward using compress_hyper_2_9_chunk__compressed_hypertable_2_device_id_dev on compress_hyper_2_9_chunk compress_hyper_2_9_chunk_1 (actual rows=0 loops=388)
Index Cond: ((device_id = d.device_id) AND (device_id_peer = 3))
-> Custom Scan (DecompressChunk) on _hyper_1_3_chunk m_4 (actual rows=0 loops=389)
-> Index Scan Backward using compress_hyper_2_8_chunk__compressed_hypertable_2_device_id_dev on compress_hyper_2_8_chunk compress_hyper_2_8_chunk_1 (actual rows=0 loops=389)
-> Custom Scan (DecompressChunk) on _hyper_1_3_chunk m_4 (actual rows=0 loops=388)
-> Index Scan Backward using compress_hyper_2_8_chunk__compressed_hypertable_2_device_id_dev on compress_hyper_2_8_chunk compress_hyper_2_8_chunk_1 (actual rows=0 loops=388)
Index Cond: ((device_id = d.device_id) AND (device_id_peer = 3))
-> Custom Scan (DecompressChunk) on _hyper_1_2_chunk m_5 (actual rows=0 loops=305)
-> Index Scan Backward using compress_hyper_2_7_chunk__compressed_hypertable_2_device_id_dev on compress_hyper_2_7_chunk compress_hyper_2_7_chunk_1 (actual rows=0 loops=305)
-> Custom Scan (DecompressChunk) on _hyper_1_2_chunk m_5 (actual rows=0 loops=304)
-> Index Scan Backward using compress_hyper_2_7_chunk__compressed_hypertable_2_device_id_dev on compress_hyper_2_7_chunk compress_hyper_2_7_chunk_1 (actual rows=0 loops=304)
Index Cond: ((device_id = d.device_id) AND (device_id_peer = 3))
-> Custom Scan (DecompressChunk) on _hyper_1_1_chunk m_6 (actual rows=0 loops=305)
-> Index Scan Backward using compress_hyper_2_6_chunk__compressed_hypertable_2_device_id_dev on compress_hyper_2_6_chunk compress_hyper_2_6_chunk_1 (actual rows=0 loops=305)
-> Custom Scan (DecompressChunk) on _hyper_1_1_chunk m_6 (actual rows=0 loops=304)
-> Index Scan Backward using compress_hyper_2_6_chunk__compressed_hypertable_2_device_id_dev on compress_hyper_2_6_chunk compress_hyper_2_6_chunk_1 (actual rows=0 loops=304)
Index Cond: ((device_id = d.device_id) AND (device_id_peer = 3))
(48 rows)

Expand Down Expand Up @@ -389,20 +399,20 @@ WHERE extract(minute FROM d.time) = 0;
-> Custom Scan (ChunkAppend) on metrics_ordered_idx m_1 (actual rows=0 loops=389)
Order: m_1."time" DESC
Hypertables excluded during runtime: 0
-> Custom Scan (DecompressChunk) on _hyper_1_5_chunk m_2 (actual rows=0 loops=389)
-> Index Scan Backward using compress_hyper_2_10_chunk__compressed_hypertable_2_device_id_de on compress_hyper_2_10_chunk compress_hyper_2_10_chunk_1 (actual rows=0 loops=389)
-> Custom Scan (DecompressChunk) on _hyper_1_5_chunk m_2 (actual rows=0 loops=388)
-> Index Scan Backward using compress_hyper_2_10_chunk__compressed_hypertable_2_device_id_de on compress_hyper_2_10_chunk compress_hyper_2_10_chunk_1 (actual rows=0 loops=388)
Index Cond: ((device_id = d.device_id) AND (device_id_peer = 3))
-> Custom Scan (DecompressChunk) on _hyper_1_4_chunk m_3 (actual rows=0 loops=389)
-> Index Scan Backward using compress_hyper_2_9_chunk__compressed_hypertable_2_device_id_dev on compress_hyper_2_9_chunk compress_hyper_2_9_chunk_1 (actual rows=0 loops=389)
-> Custom Scan (DecompressChunk) on _hyper_1_4_chunk m_3 (actual rows=0 loops=388)
-> Index Scan Backward using compress_hyper_2_9_chunk__compressed_hypertable_2_device_id_dev on compress_hyper_2_9_chunk compress_hyper_2_9_chunk_1 (actual rows=0 loops=388)
Index Cond: ((device_id = d.device_id) AND (device_id_peer = 3))
-> Custom Scan (DecompressChunk) on _hyper_1_3_chunk m_4 (actual rows=0 loops=389)
-> Index Scan Backward using compress_hyper_2_8_chunk__compressed_hypertable_2_device_id_dev on compress_hyper_2_8_chunk compress_hyper_2_8_chunk_1 (actual rows=0 loops=389)
-> Custom Scan (DecompressChunk) on _hyper_1_3_chunk m_4 (actual rows=0 loops=388)
-> Index Scan Backward using compress_hyper_2_8_chunk__compressed_hypertable_2_device_id_dev on compress_hyper_2_8_chunk compress_hyper_2_8_chunk_1 (actual rows=0 loops=388)
Index Cond: ((device_id = d.device_id) AND (device_id_peer = 3))
-> Custom Scan (DecompressChunk) on _hyper_1_2_chunk m_5 (actual rows=0 loops=305)
-> Index Scan Backward using compress_hyper_2_7_chunk__compressed_hypertable_2_device_id_dev on compress_hyper_2_7_chunk compress_hyper_2_7_chunk_1 (actual rows=0 loops=305)
-> Custom Scan (DecompressChunk) on _hyper_1_2_chunk m_5 (actual rows=0 loops=304)
-> Index Scan Backward using compress_hyper_2_7_chunk__compressed_hypertable_2_device_id_dev on compress_hyper_2_7_chunk compress_hyper_2_7_chunk_1 (actual rows=0 loops=304)
Index Cond: ((device_id = d.device_id) AND (device_id_peer = 3))
-> Custom Scan (DecompressChunk) on _hyper_1_1_chunk m_6 (actual rows=0 loops=305)
-> Index Scan Backward using compress_hyper_2_6_chunk__compressed_hypertable_2_device_id_dev on compress_hyper_2_6_chunk compress_hyper_2_6_chunk_1 (actual rows=0 loops=305)
-> Custom Scan (DecompressChunk) on _hyper_1_1_chunk m_6 (actual rows=0 loops=304)
-> Index Scan Backward using compress_hyper_2_6_chunk__compressed_hypertable_2_device_id_dev on compress_hyper_2_6_chunk compress_hyper_2_6_chunk_1 (actual rows=0 loops=304)
Index Cond: ((device_id = d.device_id) AND (device_id_peer = 3))
(44 rows)

Expand Down