fix(optimizer)!: Preserve DPipe in simplify_concat #3317
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes TobikoData/sqlmesh#2439
The
simplify_concat
rule would merge a list of binaryexp.DPipe
concatenations into oneexp.Concat
function call while constant folding any running string literals on the way. That (optimized) expression would then be generated asCONCAT(a, b, ...)
which might not have the same semantics asa || b
in every dialect e.g.tsvector
in Postgres.This PR aims to preserve
exp.DPipe
by tagging anyexp.Concat
expressions generated bysimplify_concat
withdpipe_source=True
, which will cause them to be generated back as a list of (now optimized) binaryexp.DPipe
concatenations