Skip to content

Commit

Permalink
Fix SEGMENTBY columns predicates to be pushed down
Browse files Browse the repository at this point in the history 
WHERE clause with SEGMENTBY column of type text/bytea
non-equality operators are not pushed down to Seq Scan
node of compressed chunk. This patch fixes this issue.

Fixes #5286
  • Loading branch information
@sb230132
sb230132 committed Mar 8, 2023
1 parent c76a0cf commit f54dd7b
Show file tree
Hide file tree
Showing 17 changed files with 981 additions and 534 deletions.
2 changes: 1 addition & 1 deletion CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ accidentally triggering the load of a previous DB version.**
* #5361 Add parallel support for partialize_agg()

**Bugfixes**
* #5396 Fix SEGMENTBY columns predicates to be pushed down

## 2.10.1 (2023-03-07)

Expand Down Expand Up @@ -2775,4 +2776,3 @@ the next release.
* [72f754a] use PostgreSQL's own `hash_any` function as default partfunc (thanks @robin900)
* [39f4c0f] Remove sample data instructions and point to docs site
* [9015314] Revised the `get_general_index_definition` function to handle cases where indexes have definitions other than just `CREATE INDEX` (thanks @bricklen)

1 change: 1 addition & 0 deletions tsl/src/nodes/decompress_chunk/qual_pushdown.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -357,6 +357,7 @@ modify_expression(Node *node, QualPushdownContext *context)
/* opexpr will still be checked for segment by columns */
break;
}
case T_CoerceViaIO:
case T_RelabelType:
case T_ScalarArrayOpExpr:
case T_List:
Expand Down
80 changes: 80 additions & 0 deletions tsl/test/expected/compression_qualpushdown.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -262,3 +262,83 @@ EXPLAIN (costs off) SELECT * FROM pushdown_relabel WHERE dev_vc = 'varchar'::cha
Filter: ((dev_vc)::bpchar = 'varchar '::character(10))
(4 rows)

-- github issue #5286
CREATE TABLE deleteme AS
SELECT generate_series AS timestamp, 1 AS segment, 0 AS data
FROM generate_series('2008-03-01 00:00'::timestamp with time zone,
'2008-03-04 12:00', '1 second');
SELECT create_hypertable('deleteme', 'timestamp', migrate_data => true);
NOTICE: adding not-null constraint to column "timestamp"
NOTICE: migrating data to chunks
create_hypertable
-----------------------
(7,public,deleteme,t)
(1 row)

ALTER TABLE deleteme SET (
timescaledb.compress,
timescaledb.compress_segmentby = 'segment'
);
SELECT compress_chunk(i) FROM show_chunks('deleteme') i;
compress_chunk
----------------------------------------
_timescaledb_internal._hyper_7_8_chunk
(1 row)

EXPLAIN (costs off) SELECT sum(data) FROM deleteme WHERE segment::text like '%4%';
QUERY PLAN
---------------------------------------------------------
Aggregate
-> Custom Scan (DecompressChunk) on _hyper_7_8_chunk
-> Seq Scan on compress_hyper_8_9_chunk
Filter: ((segment)::text ~~ '%4%'::text)
(4 rows)

EXPLAIN (costs off) SELECT sum(data) FROM deleteme WHERE '4' = segment::text;
QUERY PLAN
---------------------------------------------------------
Aggregate
-> Custom Scan (DecompressChunk) on _hyper_7_8_chunk
-> Seq Scan on compress_hyper_8_9_chunk
Filter: ('4'::text = (segment)::text)
(4 rows)

CREATE TABLE deleteme_with_bytea(time bigint NOT NULL, bdata bytea);
SELECT create_hypertable('deleteme_with_bytea', 'time', chunk_time_interval => 1000000);
create_hypertable
----------------------------------
(9,public,deleteme_with_bytea,t)
(1 row)

INSERT INTO deleteme_with_bytea(time, bdata) VALUES (1001, E'\\x');
INSERT INTO deleteme_with_bytea(time, bdata) VALUES (1001, NULL);
ALTER TABLE deleteme_with_bytea SET (
timescaledb.compress,
timescaledb.compress_segmentby = 'bdata'
);
SELECT compress_chunk(i) FROM show_chunks('deleteme_with_bytea') i;
compress_chunk
-----------------------------------------
_timescaledb_internal._hyper_9_10_chunk
(1 row)

EXPLAIN (costs off) SELECT '1' FROM deleteme_with_bytea WHERE bdata = E'\\x';
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------
Result
-> Custom Scan (DecompressChunk) on _hyper_9_10_chunk
-> Index Scan using compress_hyper_10_11_chunk__compressed_hypertable_10_bdata__ts_ on compress_hyper_10_11_chunk
Index Cond: (bdata = '\x'::bytea)
(4 rows)

EXPLAIN (costs off) SELECT '1' FROM deleteme_with_bytea WHERE bdata::text = '123';
QUERY PLAN
----------------------------------------------------------
Result
-> Custom Scan (DecompressChunk) on _hyper_9_10_chunk
-> Seq Scan on compress_hyper_10_11_chunk
Filter: ((bdata)::text = '123'::text)
(4 rows)

DROP table deleteme;
DROP table deleteme_with_bytea;
22 changes: 14 additions & 8 deletions tsl/test/expected/transparent_decompression-12.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1505,7 +1505,8 @@ ORDER BY time,
-> Custom Scan (DecompressChunk) on _hyper_1_3_chunk (actual rows=2520 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
-> Seq Scan on compress_hyper_5_16_chunk (actual rows=5 loops=1)
(10 rows)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
(11 rows)

-- test aggregate
:PREFIX
Expand Down Expand Up @@ -4920,16 +4921,19 @@ ORDER BY time,
-> Merge Append (actual rows=0 loops=1)
-> Custom Scan (DecompressChunk) on _hyper_2_4_chunk (actual rows=0 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 360
-> Seq Scan on compress_hyper_6_17_chunk (actual rows=1 loops=1)
-> Seq Scan on compress_hyper_6_17_chunk (actual rows=0 loops=1)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 1
-> Custom Scan (DecompressChunk) on _hyper_2_5_chunk (actual rows=0 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 1080
-> Seq Scan on compress_hyper_6_18_chunk (actual rows=3 loops=1)
-> Seq Scan on compress_hyper_6_18_chunk (actual rows=0 loops=1)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 3
-> Custom Scan (DecompressChunk) on _hyper_2_6_chunk (actual rows=0 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 360
-> Seq Scan on compress_hyper_6_19_chunk (actual rows=1 loops=1)
-> Seq Scan on compress_hyper_6_19_chunk (actual rows=0 loops=1)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 1
-> Merge Append (actual rows=1675 loops=1)
-> Seq Scan on _hyper_2_7_chunk (actual rows=335 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
Expand All @@ -4943,12 +4947,14 @@ ORDER BY time,
-> Custom Scan (DecompressChunk) on _hyper_2_10_chunk (actual rows=504 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
-> Seq Scan on compress_hyper_6_20_chunk (actual rows=1 loops=1)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
-> Custom Scan (DecompressChunk) on _hyper_2_11_chunk (actual rows=1512 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
-> Seq Scan on compress_hyper_6_21_chunk (actual rows=3 loops=1)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
-> Seq Scan on _hyper_2_12_chunk (actual rows=504 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
(35 rows)
(40 rows)

-- test aggregate
:PREFIX
Expand Down
22 changes: 14 additions & 8 deletions tsl/test/expected/transparent_decompression-13.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1643,7 +1643,8 @@ ORDER BY time,
-> Custom Scan (DecompressChunk) on _hyper_1_3_chunk (actual rows=2520 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
-> Seq Scan on compress_hyper_5_16_chunk (actual rows=5 loops=1)
(15 rows)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
(16 rows)

-- test aggregate
:PREFIX
Expand Down Expand Up @@ -5735,8 +5736,9 @@ ORDER BY time,
Sort Method: quicksort
-> Custom Scan (DecompressChunk) on _hyper_2_4_chunk (actual rows=0 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 360
-> Seq Scan on compress_hyper_6_17_chunk (actual rows=1 loops=1)
-> Seq Scan on compress_hyper_6_17_chunk (actual rows=0 loops=1)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 1
-> Sort (actual rows=0 loops=1)
Sort Key: _hyper_2_5_chunk."time"
Sort Method: quicksort
Expand All @@ -5745,8 +5747,9 @@ ORDER BY time,
Sort Method: quicksort
-> Custom Scan (DecompressChunk) on _hyper_2_5_chunk (actual rows=0 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 1080
-> Seq Scan on compress_hyper_6_18_chunk (actual rows=3 loops=1)
-> Seq Scan on compress_hyper_6_18_chunk (actual rows=0 loops=1)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 3
-> Sort (actual rows=0 loops=1)
Sort Key: _hyper_2_6_chunk."time"
Sort Method: quicksort
Expand All @@ -5755,8 +5758,9 @@ ORDER BY time,
Sort Method: quicksort
-> Custom Scan (DecompressChunk) on _hyper_2_6_chunk (actual rows=0 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 360
-> Seq Scan on compress_hyper_6_19_chunk (actual rows=1 loops=1)
-> Seq Scan on compress_hyper_6_19_chunk (actual rows=0 loops=1)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 1
-> Merge Append (actual rows=1675 loops=1)
Sort Key: _hyper_2_7_chunk."time"
-> Index Scan Backward using _hyper_2_7_chunk_metrics_space_time_idx on _hyper_2_7_chunk (actual rows=335 loops=1)
Expand All @@ -5776,6 +5780,7 @@ ORDER BY time,
-> Custom Scan (DecompressChunk) on _hyper_2_10_chunk (actual rows=504 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
-> Seq Scan on compress_hyper_6_20_chunk (actual rows=1 loops=1)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
-> Sort (actual rows=1512 loops=1)
Sort Key: _hyper_2_11_chunk."time"
Sort Method: quicksort
Expand All @@ -5785,9 +5790,10 @@ ORDER BY time,
-> Custom Scan (DecompressChunk) on _hyper_2_11_chunk (actual rows=1512 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
-> Seq Scan on compress_hyper_6_21_chunk (actual rows=3 loops=1)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
-> Index Scan Backward using _hyper_2_12_chunk_metrics_space_time_idx on _hyper_2_12_chunk (actual rows=504 loops=1)
Index Cond: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
(68 rows)
(73 rows)

-- test aggregate
:PREFIX
Expand Down
22 changes: 14 additions & 8 deletions tsl/test/expected/transparent_decompression-14.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1643,7 +1643,8 @@ ORDER BY time,
-> Custom Scan (DecompressChunk) on _hyper_1_3_chunk (actual rows=2520 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
-> Seq Scan on compress_hyper_5_16_chunk (actual rows=5 loops=1)
(15 rows)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
(16 rows)

-- test aggregate
:PREFIX
Expand Down Expand Up @@ -5735,8 +5736,9 @@ ORDER BY time,
Sort Method: quicksort
-> Custom Scan (DecompressChunk) on _hyper_2_4_chunk (actual rows=0 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 360
-> Seq Scan on compress_hyper_6_17_chunk (actual rows=1 loops=1)
-> Seq Scan on compress_hyper_6_17_chunk (actual rows=0 loops=1)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 1
-> Sort (actual rows=0 loops=1)
Sort Key: _hyper_2_5_chunk."time"
Sort Method: quicksort
Expand All @@ -5745,8 +5747,9 @@ ORDER BY time,
Sort Method: quicksort
-> Custom Scan (DecompressChunk) on _hyper_2_5_chunk (actual rows=0 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 1080
-> Seq Scan on compress_hyper_6_18_chunk (actual rows=3 loops=1)
-> Seq Scan on compress_hyper_6_18_chunk (actual rows=0 loops=1)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 3
-> Sort (actual rows=0 loops=1)
Sort Key: _hyper_2_6_chunk."time"
Sort Method: quicksort
Expand All @@ -5755,8 +5758,9 @@ ORDER BY time,
Sort Method: quicksort
-> Custom Scan (DecompressChunk) on _hyper_2_6_chunk (actual rows=0 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 360
-> Seq Scan on compress_hyper_6_19_chunk (actual rows=1 loops=1)
-> Seq Scan on compress_hyper_6_19_chunk (actual rows=0 loops=1)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
Rows Removed by Filter: 1
-> Merge Append (actual rows=1675 loops=1)
Sort Key: _hyper_2_7_chunk."time"
-> Index Scan Backward using _hyper_2_7_chunk_metrics_space_time_idx on _hyper_2_7_chunk (actual rows=335 loops=1)
Expand All @@ -5776,6 +5780,7 @@ ORDER BY time,
-> Custom Scan (DecompressChunk) on _hyper_2_10_chunk (actual rows=504 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
-> Seq Scan on compress_hyper_6_20_chunk (actual rows=1 loops=1)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
-> Sort (actual rows=1512 loops=1)
Sort Key: _hyper_2_11_chunk."time"
Sort Method: quicksort
Expand All @@ -5785,9 +5790,10 @@ ORDER BY time,
-> Custom Scan (DecompressChunk) on _hyper_2_11_chunk (actual rows=1512 loops=1)
Filter: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
-> Seq Scan on compress_hyper_6_21_chunk (actual rows=3 loops=1)
Filter: (_ts_meta_max_3 > ('2000-01-08'::cstring)::timestamp with time zone)
-> Index Scan Backward using _hyper_2_12_chunk_metrics_space_time_idx on _hyper_2_12_chunk (actual rows=504 loops=1)
Index Cond: ("time" > ('2000-01-08'::cstring)::timestamp with time zone)
(68 rows)
(73 rows)

-- test aggregate
:PREFIX
Expand Down
Loading

0 comments on commit f54dd7b

Please sign in to comment.