`optimize_inverse_dictionary_lookup` rewrite inside a correlated EXISTS makes decorrelation fail: 48 "Cannot decorrelate query, because 'DelayedCreatingSets' step is not supported"

[zlareb1]

Describe the problem

A valid query with a correlated EXISTS subquery fails with exception 48 (NOT_IMPLEMENTED) at pure default settings when the subquery's WHERE contains a dictGet(...) >= <const> comparison.

optimize_inverse_dictionary_lookup (default 1) rewrites the dictGet('d', 'attr', key) >= c predicate into key IN __set_.... When the predicate sits inside a correlated subquery, the injected set adds a DelayedCreatingSets step to the subquery plan, and the correlated-subquery decorrelation then refuses the plan:

Code: 48. DB::Exception: Cannot decorrelate query, because 'DelayedCreatingSets' step is not supported. (NOT_IMPLEMENTED)

Both settings involved are on by default (optimize_inverse_dictionary_lookup = 1, allow_experimental_correlated_subqueries = 1), so an ordinary query errors out of the box. Disabling the rewrite (optimize_inverse_dictionary_lookup = 0) makes the same query run fine and return the correct result. This is the only setting that matters: correlated_subqueries_use_in_memory_buffer = 0 does not help, and the same failure occurs when the EXISTS is used as a scalar (e.g. o.g >= (EXISTS ...)), not just in WHERE.

How to reproduce

Version: 26.8.1.653 (public master build; also reproduced on 26.8.1.561). All settings at defaults.

CREATE TABLE i_src (id UInt64, val UInt32) ENGINE=MergeTree ORDER BY id;
INSERT INTO i_src SELECT number, number % 97 FROM numbers(500);

CREATE DICTIONARY i_dict (id UInt64, val UInt32 DEFAULT 0) PRIMARY KEY id
  SOURCE(CLICKHOUSE(TABLE 'i_src' DB 'default')) LIFETIME(0) LAYOUT(FLAT());

CREATE TABLE i_t (k UInt32, g Int32, s String) ENGINE=MergeTree ORDER BY k;
INSERT INTO i_t SELECT number, number % 7, toString(number % 5) FROM numbers(100);

SELECT o.k FROM i_t AS o WHERE EXISTS (
    SELECT 1 FROM i_t AS i
    WHERE i.s = o.s AND dictGet('i_dict', 'val', toUInt64(abs(g)) % 500) >= 76);

Observed (deterministic, 20/20 runs):

Code: 48. DB::Exception: Cannot decorrelate query, because 'DelayedCreatingSets' step is not supported. (NOT_IMPLEMENTED)

Expected: the query executes and returns the rows for which the EXISTS holds (here an empty result — confirmed by running with SETTINGS optimize_inverse_dictionary_lookup = 0, which succeeds).

Settings analysis:

• REQUIRED: none beyond defaults. optimize_inverse_dictionary_lookup = 0 is the only toggle that avoids the error.
• Incidental: correlated_subqueries_use_in_memory_buffer (both values fail), the exact comparison constant, table sizes, dictionary layout.

Additional context

Mechanism: InverseDictionaryLookupPass rewrites the dictGet comparison into a membership test against an internally built set (__set_...). Building that set inside the correlated subquery introduces a DelayedCreatingSets plan step, and the decorrelation of correlated subqueries does not support that step, so planning aborts with 48. Either the pass should be skipped inside correlated subqueries it would break, or the decorrelator should learn to handle DelayedCreatingSets.

Same "optimization-pass output escapes into an unsupported context" family as the ColumnSet-reaches-FINAL-merge manifestation of the same pass, but a distinct trigger and error code.

Related: #112030 Related: #99500