Improve compressed DML expression pushdown #6895
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #6895 +/- ##
==========================================
+ Coverage 80.06% 80.73% +0.66%
==========================================
Files 190 199 +9
Lines 37181 37209 +28
Branches 9450 9726 +276
==========================================
+ Hits 29770 30041 +271
- Misses 2997 3206 +209
+ Partials 4414 3962 -452 ☔ View full report in Codecov by Sentry. |
svenklemm
force-pushed
the
compressed_dml_constify
branch
from
14f4fd6
to
2be3ae9
Compare
| */ | ||
| static void | ||
| process_predicates(Chunk *ch, CompressionSettings *settings, List *predicates, List **heap_filters, | ||
| List **index_filters, List **is_null) | ||
| { | ||
| ListCell *lc; | ||
| PlannerGlobal glob = { .boundParams = NULL }; |
There was a problem hiding this comment.
Should we take es_param_list_info from the node state here, like in runtime chunk exclusion? Then it could work for prepared statement params, and under joins.
There was a problem hiding this comment.
I think for joins it would not be safe as we dont repeat decompression in rescan. For now this is intentional. I'll add some more tests cause having prepared statement support would be nice.
There was a problem hiding this comment.
Oh, right, I think it's not even possible, because the rescans happen underneath the DELETE, and it receives the tids only. And we have to guess which batches we have to decompress even before the underlying nested loop starts running.
Would be nice if the underlying scans could return the matching compressed tids as well, and then we'd use that to decompress, but that's a completely different approach.
There was a problem hiding this comment.
Might also create additional weirdness with the snapshots betweeen the internal scan and the normal scan
There was a problem hiding this comment.
Prepared Statement works too as they are handled differently from join params, i'll add a test
There was a problem hiding this comment.
I've added a comment as well.
| @@ -2565,7 +2569,11 @@ process_predicates(Chunk *ch, CompressionSettings *settings, List *predicates, L | |||
| continue; | |||
|
|
|||
| if (!IsA(expr, Const)) | |||
| continue; | |||
| { | |||
| expr = (Expr *) estimate_expression_value(&root, (Node *) expr); | |||
There was a problem hiding this comment.
Currently this means we're able to evaluate the stable functions in WHERE conditions, when it is on one side of a supported operator, right?
There was a problem hiding this comment.
correct, all the sqlvaluefunction (current_timestamp, current_user, ...) will also be supported with this
svenklemm
force-pushed
the
compressed_dml_constify
branch
5 times, most recently
from
10d6685
to
54c5d04
Compare
svenklemm
force-pushed
the
compressed_dml_constify
branch
from
54c5d04
to
16c077e
Compare
Try to constify query constraints when filtering batches to cover a wider range of expressions that are safe to evaluate when we do the batch filtering. This will allow the following expressions to be usable with compressed DML batch filtering: - IMMUTABLE and STABLE functions - SQLValueFunction (CURRENT_TIME,CURRENT_USER,LOCAL_TIME,...) - Prepared Statement parameters
svenklemm
force-pushed
the
compressed_dml_constify
branch
from
16c077e
to
19d05ec
Compare
This release contains performance improvements and bug fixes since the 2.15.3 release. We recommend that you upgrade at the next available opportunity. **Features** * timescale#6880: Add support for the array operators used for compressed DML batch filtering. * timescale#6895: Improve the compressed DML expression pushdown. * timescale#6897: Add support for replica identity on compressed hypertables. * timescale#6918: Remove support for PG13. * timescale#6920: Rework compression activity wal markers. * timescale#6989: Add support for foreign keys when converting plain tables to hypertables. * timescale#7020: Add support for the chunk column statistics tracking. * timescale#7048: Add an index scan for INSERT DML decompression. * timescale#7075: Reduce decompression on the compressed INSERT. * timescale#7101: Reduce decompressions for the compressed UPDATE/DELETE. * timescale#7108 Reduce decompressions for INSERTs with UNIQUE constraints **Bugfixes** * timescale#7018: Fix `search_path` quoting in the compression defaults function. * timescale#7046: Prevent locking for compressed tuples. * timescale#7055: Fix the `scankey` for `segment by` columns, where the type `constant` is different to `variable`. * timescale#7064: Fix the bug in the default `order by` calculation in compression. * timescale#7069: Fix the index column name usage. * timescale#7074: Fix the bug in the default `segment by` calculation in compression. **Thanks**
Try to constify query constraints when filtering batches to cover
a wider range of expressions that are safe to evaluate when we
do the batch filtering.
This will allow the following expressions to be usable with
compressed DML batch filtering: