Optimize compressed chunk resorting

[jnidzwetzki]

This patch adds an optimization to the DecompressChunk node. If the query 'order by' and the compression 'order by' are compatible (query 'order by' is equal or a prefix of compression 'order by'), the compressed batches of the segments are decompressed in parallel and merged using a binary heep. This preserves the ordering and the sorting of the result can be prevented. Especially LIMIT queries benefit from this optimization because only the first tuples of some batches have to be decompressed. Previously, all segments were completely decompressed and sorted.

Single Query Execution Directly in PostgreSQL

--- New
test2=# SELECT * FROM sensor_data ORDER BY time DESC LIMIT 1;
             time              | sensor_id |        cpu         |   temperature    
-------------------------------+-----------+--------------------+------------------
 2023-03-22 16:22:08.119032+01 |        53 | 0.7768845568255323 | 95.6922406819352
(1 row)

Time: 10.486 ms

--- Old
test2=# SET timescaledb.enable_decompression_heap_merge = 0;
SET
Time: 2.303 ms
test2=# SELECT * FROM sensor_data ORDER BY time DESC LIMIT 1;
             time              | sensor_id |        cpu         |   temperature    
-------------------------------+-----------+--------------------+------------------
 2023-03-22 16:22:08.119032+01 |        53 | 0.7768845568255323 | 95.6922406819352
(1 row)

Time: 560.409 ms

TSbench

$  tsbench -vvvv --with-connection pgsq://localhost:5432/benchmark  --benchmarks 'ordered_append_compressed'

Report for benchmark suite 'compressed_chunk_order'
+--------------------------------------------------------------------+------------------------------------------+
| Query                                                              | 0e8177f8aadb162ebb94abd171c041761e2bbe55 |
+--------------------------------------------------------------------+------------------------------------------+
| SELECT * FROM sensor_data_compressed ORDER BY time ASC LIMIT 1;    |                                   154.23 |
| SELECT * FROM sensor_data_compressed ORDER BY time ASC LIMIT 100;  |                                   154.00 |
| SELECT * FROM sensor_data_compressed ORDER BY time ASC;            |                                  2322.28 |
| SELECT * FROM sensor_data_compressed ORDER BY time DESC LIMIT 1;   |                                     5.94 |
| SELECT * FROM sensor_data_compressed ORDER BY time DESC LIMIT 100; |                                    12.59 |
| SELECT * FROM sensor_data_compressed ORDER BY time DESC;           |                                  2184.67 |
+--------------------------------------------------------------------+------------------------------------------+

Note: For ORDER BY time DESC queries, the merge optimization is enabled. The ORDER BY time ASC queries use the regular query plans.

Comparison with execution times on 2.10.1

$  tsbench -vvvv --with-connection pgsq://localhost:5432/benchmark  --benchmarks 'ordered_append_compressed'

Report for benchmark suite 'compressed_chunk_order'
+--------------------------------------------------------------------+------------------------------------------+
| Query                                                              | 540a63e6788e8b66cd86099d5c17f50507dcd080 |
+--------------------------------------------------------------------+------------------------------------------+
| SELECT * FROM sensor_data_compressed ORDER BY time ASC LIMIT 1;    |                                   156.90 |
| SELECT * FROM sensor_data_compressed ORDER BY time ASC LIMIT 100;  |                                   156.72 |
| SELECT * FROM sensor_data_compressed ORDER BY time ASC;            |                                  2341.44 |
| SELECT * FROM sensor_data_compressed ORDER BY time DESC LIMIT 1;   |                                   160.72 |
| SELECT * FROM sensor_data_compressed ORDER BY time DESC LIMIT 100; |                                   158.97 |
| SELECT * FROM sensor_data_compressed ORDER BY time DESC;           |                                  2303.65 |
+--------------------------------------------------------------------+------------------------------------------+

So, this PR does not affect the performance of the existing code path and just adds an optimization for certain queries.

Fixes: #4223

[codecov]

Codecov Report

Merging #5530 (7f661df) into main (cc9c3b3) will decrease coverage by 0.29%.
The diff coverage is 96.31%.

❗ Current head 7f661df differs from pull request most recent head 8a794df. Consider uploading reports for the commit 8a794df to get more accurate results

@@            Coverage Diff             @@
##             main    #5530      +/-   ##
==========================================
- Coverage   90.92%   90.64%   -0.29%     
==========================================
  Files         229      229              
  Lines       54064    54141      +77     
==========================================
- Hits        49158    49076      -82     
- Misses       4906     5065     +159     

Impacted Files	Coverage Δ
src/compat/compat.h	96.61% <ø> (+6.13%)	⬆️
tsl/src/nodes/decompress_chunk/decompress_chunk.c	94.19% <87.17%> (+0.19%)	⬆️
tsl/src/nodes/decompress_chunk/planner.c	92.25% <92.75%> (-0.14%)	⬇️
tsl/src/nodes/decompress_chunk/exec.c	96.32% <99.08%> (+2.69%)	⬆️
src/guc.c	95.38% <100.00%> (+0.07%)	⬆️
src/import/planner.c	65.15% <100.00%> (ø)
tsl/src/compression/create.c	96.30% <100.00%> (ø)

... and 39 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[akuzm]

I don't really like the interface we ended up with, feels a little confusing. Anyway, let's proceed with it, we can improve later. Would be good to see some tsbench benchmarks.