Skip to content

Support date-and-time string literals in IN for Date columns - #111011

Open
Onyx2406 wants to merge 1 commit into
ClickHouse:masterfrom
Onyx2406:fix-date-in-datetime-strings
Open

Support date-and-time string literals in IN for Date columns#111011
Onyx2406 wants to merge 1 commit into
ClickHouse:masterfrom
Onyx2406:fix-date-in-datetime-strings

Conversation

@Onyx2406

Copy link
Copy Markdown
Contributor

Related: #111000

d IN ('2026-01-01 00:00:00') on a Date column threw TYPE_MISMATCH, while the equivalent comparison d = '2026-01-01 00:00:00' works with #111000. This PR extends the same semantics to IN/NOT IN set building. Best merged after #111000 (independent code, but the two together make string handling consistent across = and IN).

Mechanism

All literal IN sets funnel through one conversion point (convertFieldToTypeCheckEnum in src/Analyzer/SetUtils.cpp, used by the analyzer, the planner and the legacy path). Since IN is pure equality, "compare through the DateTime64 supertype" is exactly equivalent to: keep a string element iff it parses as DateTime64(6) and is the exact midnight of a day representable in the column type; otherwise the element can never equal any Date value and is excluded from the set (the same principle the strict-IN machinery already applies to non-representable numeric literals). The set stays Date-typed, so index analysis and set indexes work unchanged. Unparsable strings throw the same errors as before.

toDate('2026-01-01') IN ('2026-01-01 00:00:00') → 1, IN ('2026-01-01 12:00:00') → 0, NOT IN accordingly.

Disclosed asymmetries (pre-existing, deliberately out of scope)

  • Typed DateTime elements truncate-match on master: d IN (toDateTime('... 12:00:00')) returns 1 (DateTime→Date truncation in convertFieldToType), contradicting both d = toDateTime('... 12:00:00') (0) and — after this PR — the string spelling (0). Changing that would alter long-standing behavior of typed elements and deserves its own discussion.
  • String elements here parse with default format settings (like all IN literals), while = honors the session's date_time_input_format after Respect format settings when parsing query parameter values #110985/Support comparing Date with a String containing a date and time #111000 — a value parsable only with best_effort still throws in IN.
  • Single-element tuple form tuple(d) IN (('...',)) takes the generic tuple conversion and still throws — same structural gap that already exists for enum literals in tuples.
  • A datetime string on the left of IN against a Date set is likewise untouched.

Verified by code-path analysis plus the stateless test 04627_date_in_datetime_strings (midnight/noon, fractional seconds, NOT IN, Date32 incl. pre-epoch, out-of-range dates, mixed lists, Nullable, MergeTree key filtering, unchanged error cases; session timezone pinned); no local build was run, so correctness relies on CI.

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

IN with string literals containing a date and time now works for Date and Date32 columns, consistent with the comparison operators.

A follow-up to ClickHouse#111000, which made
comparison operators support comparing `Date`/`Date32` with a string containing
a date and time, e.g. `d = '2026-01-01 00:00:00'`. The `IN` operator still threw
`TYPE_MISMATCH` for `d IN ('2026-01-01 00:00:00')`.

Now such a string element of an `IN` set is parsed as `DateTime64` with the
scale 6, consistently with comparison operators, and it is kept in the set only
if it is exactly the midnight of a date representable in the column type. Any
other value (e.g. `'2026-01-01 12:00:00'`) cannot be equal to any value of a
`Date`/`Date32` column, so it is excluded from the set, like other values that
do not represent any possible value of the type of the filtered column. Strings
that are parsable neither as a date nor as a date with a time still throw.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant