Skip to content

Translate standard DateTime members and methods (#55) - #64

Draft
alex-clickhouse wants to merge 2 commits into
fix/issue-53-datetimeoffsetfrom
feature/issue-55-datetime-translators
Draft

Translate standard DateTime members and methods (#55)#64
alex-clickhouse wants to merge 2 commits into
fix/issue-53-datetimeoffsetfrom
feature/issue-55-datetime-translators

Conversation

@alex-clickhouse

Copy link
Copy Markdown
Collaborator

Fixes #55.

Stacked on #63. The base branch is fix/issue-53-datetimeoffset, not main. Review #63 first,
and read this diff as the second commit only. The stack matters: #63 supplies the DateTimeOffset
store mapping, which is what lets these translators cover DateTimeOffset as the issue asked.

Problem

The provider registered no date/time member translator. EF Core's base
RelationalMemberTranslatorProvider adds none of its own, so only a direct comparison worked and
every member and method threw The LINQ expression ... could not be translated. Subtracting one date
from another was worse: it failed with an internal cast or coercion error naming CLR types the user
never wrote.

Solution

One shared translator serves DateTime, DateTimeOffset and DateOnly, because the ClickHouse
function is the same for each. RegisterInstanceMembers registers only the members a given type
declares, so DateOnly gets the date components alone.

.NET ClickHouse
.Year .Month .Day toYear toMonth toDayOfMonth
.Hour .Minute .Second .Millisecond toHour toMinute toSecond toMillisecond
.DayOfYear toDayOfYear
.DayOfWeek toDayOfWeek(x, 2)
.Date toStartOfDay
.TimeOfDay toTime64(x, 7)
.AddYears(n) .AddMonths(n) addYears addMonths
.AddDays(n).AddMilliseconds(n) addDays … (see below)
DateTime.UtcNow / .Now / .Today now64(7, 'UTC') / now64(7) / toStartOfDay(now())

Every design decision below was measured against a real ClickHouse 26.7.1 server, and .NET behaviour
was measured on .NET 10 rather than taken from the documentation.

.DayOfWeek needs no arithmetic. Week mode 2 agrees with System.DayOfWeek exactly (Sunday 0
through Saturday 6). The mode argument is always sent, because the default mode 0 starts the week on
Monday.

.DayOfWeek carries a number-backed enum mapping. This provider maps a C# enum to a ClickHouse
string, so the default mapping would render x.DayOfWeek == DayOfWeek.Sunday as a comparison against
'Sunday' while the function returns a number. An EnumToNumberConverter over the Int32 mapping
fixes both sides. Verified against a server for projection, WHERE, parameters, GROUP BY,
HAVING, ORDER BY, DISTINCT and IN.

.TimeOfDay uses toTime64(x, 7), not toTime. One .NET tick is 100 ns, which is Time64
precision 7, so the fraction survives; toTime drops it.

Add* is exact or is not translated. This is the subtle part. AddDays and the other time-based
methods take a double, which .NET scales to whole ticksAddSeconds(0.1234567) adds exactly
1 234 567 ticks. The matching ClickHouse function takes a whole number of its own unit and discards
the rest, so addDays(x, 1.5) would add only one day. A constant is therefore folded to ticks and
expressed in the coarsest unit that holds it exactly:

e.Timestamp.AddDays(1)             // addDays(ts, 1)
e.Timestamp.AddDays(1.5)           // addMilliseconds(ts, 129600000)
e.Timestamp.AddMilliseconds(0.5)   // not translated — 5 000 ticks is below millisecond resolution
e.Timestamp.AddDays(someVariable)  // not translated — cannot be checked for exactness

Preferring the natural function keeps the store type of the source, and it is the only form that
works on a Date/Date32 column — ClickHouse rejects addMilliseconds on those with
ILLEGAL_TYPE_OF_ARGUMENT.

Anything that cannot be expressed exactly is left untranslated rather than rounded to fit. Three
cases: a sub-millisecond offset, a non-constant offset, and a value outside the DateTime range. An
untranslated call still gives the correct .NET value through client evaluation in a projection, and
reports a clear reason in a predicate.

Milliseconds are as fine as this goes deliberately. addNanoseconds would express a tick exactly,
but promotes the result to DateTime64(9), whose Int64 nanosecond count cannot span the
DateTime64 range — that would trade a rounding error for a silently wrong date. Server-side
rounding was also rejected: ClickHouse round() is banker's rounding (round(2.5) is 2), so it
disagrees with .NET.

Date/time arithmetic now reports a reason. ClickHouse has no operator matching the .NET result
for any of these shapes, and each failed differently before:

  • one date minus another gives a TimeSpan, whereas dateDiff counts whole units;
  • one time of day minus another gives a TimeSpan, whereas Time64 subtraction gives a Decimal
    of seconds;
  • a date plus a TimeSpan keeps the date type in .NET, whereas ClickHouse rejects the mixed
    operands.

VisitBinary now reports these through EF Core's translation-error channel instead of throwing, so
a projection falls back to the client and returns the correct value, and a predicate explains why.

Tests

DateTimeMemberTranslationTests — 60 tests. Integration tests run against a real ClickHouse through
Testcontainers, as AGENTS.md prefers, with a small offline class for SQL-shape assertions.

Coverage worth calling out: sub-millisecond Add* values, a parameterised offset, an out-of-range
offset, AddMonths day clamping, .DayOfWeek compared against a .NET constant, .TimeOfDay to one
tick, Date32 columns, TimeSpan arithmetic falling back to the client, and the DateTimeOffset
equivalents.

Also in this PR

A previously-unsupported Northwind query, GroupJoin_aggregate_anonymous_key_selectors2, now
translates. Its provider-specific override asserted InvalidOperationException, so the override is
removed and the base test runs. The functional suite goes from 321 passed + 2 failed to 323 passed.

Behaviour change

DateTime.Now and DateTime.Today in a projection used to be evaluated on the client; they now
read the server clock. The value therefore follows the server's timezone rather than the
client's, and comes back with DateTimeKind.Unspecified instead of Local. Use DateTime.UtcNow
for an instant that does not depend on server configuration. In a predicate all three were
untranslatable before, so nothing changes there.

This matches how other EF Core providers translate these members (GETDATE(), now()), but it is a
semantic shift and worth a second opinion.

Known limits

  • .Date uses toStartOfDay, which returns a DateTime spanning 1970–2106. ClickHouse wraps a
    value outside that window instead of reporting it, so .Date on a DateTime64 column holding a
    pre-1970 date reads back wrong. Enabling
    enable_extended_results_for_datetime_functions gives a range-preserving DateTime64 result —
    measured. This is the same caveat that already applies to EF.Functions.ToStartOfDay, and it is
    now documented for both.
  • Not translated: .Ticks, .AddTicks, .Microsecond/.Nanosecond, and DateTimeOffset's
    .UtcDateTime/.LocalDateTime/.Offset.
  • The ClickHouse-specific functions (dateDiff, dateTrunc) stay in Add EF.Functions translations for the remaining ClickHouse date/time functions #58.

🤖 Generated with Claude Code

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds standard .NET date/time translation support to the ClickHouse EF Core query pipeline.

Changes:

  • Translates date/time components, clocks, and Add* methods.
  • Improves unsupported date/time arithmetic handling.
  • Adds integration coverage and user documentation.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
ClickHouseDateTimeMemberTranslator.cs Translates date/time members and clocks.
ClickHouseDateTimeMethodTranslator.cs Translates Add* methods.
ClickHouseMemberTranslatorProvider.cs Registers the member translator.
ClickHouseSqlTranslatingExpressionVisitor.cs Handles unsupported date/time arithmetic.
DateTimeMemberTranslationTests.cs Adds translation and integration tests.
NorthwindJoinQueryClickHouseTest.cs Re-enables a formerly unsupported query.
README.md Documents supported translations and limits.
CHANGELOG.md Records the feature and behavior changes.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

}) ?? throw new InvalidOperationException("Method ToStartOfInterval with strict signature not found.");

RegisterAddMethods(typeof(DateTime), hasTimeComponents: true);
RegisterAddMethods(typeof(DateTimeOffset), hasTimeComponents: true);
Comment on lines +328 to +330
var scaled = value * ticksPerUnit + (value >= 0 ? 0.5 : -0.5);

return double.Abs(scaled) > DateTime.MaxValue.Ticks ? null : (long)scaled;
Comment on lines +86 to +89
// A DateTimeOffset is an instant, and its store type is UTC-pinned, so both of its clock
// members read the same UTC value.
ServerClockMembers.Add(Property(typeof(DateTimeOffset), nameof(DateTimeOffset.UtcNow)), ServerClock.UtcNow);
ServerClockMembers.Add(Property(typeof(DateTimeOffset), nameof(DateTimeOffset.Now)), ServerClock.UtcNow);
Comment on lines +284 to +286
if (value is not SqlConstantExpression { Value: double constantValue })
{
return null;
Comment thread CHANGELOG.md
* **`.TimeOfDay`** → `toTime64(x, 7)`; precision 7 is one .NET tick, so no part of the value is lost (`toTime` would drop the fraction).
* **`DateTime.UtcNow`** → `now64(7, 'UTC')`, **`DateTime.Now`** → `now64(7)` and **`DateTime.Today`** → `toStartOfDay(now())`. `today()` is not used for `.Today` because it returns a `Date`, whereas the member's type is `DateTime`.
* **`.AddYears(n)`** → `addYears` and **`.AddMonths(n)`** → `addMonths`. Both take an `int` in .NET, and ClickHouse clamps the day of month the same way .NET does, so `2026-01-31` plus one month gives `2026-02-28` in both.
* **`.AddDays`/`.AddHours`/`.AddMinutes`/`.AddSeconds`/`.AddMilliseconds`** take a `double` in .NET, which .NET scales to whole **ticks** (100 ns), rounding half away from zero — so `AddSeconds(0.1234567)` adds exactly 1 234 567 ticks. The matching ClickHouse function takes a whole number of its own unit and discards the rest, so `addDays(x, 1.5)` would add only one day. A constant argument is therefore folded to ticks during translation and then expressed in the coarsest unit that holds it exactly: a whole number of the unit emits the natural function (`addDays(x, 1)`), and otherwise `addMilliseconds` carries the exact count (`AddDays(1.5)` → `addMilliseconds(x, 129600000)`). Preferring the natural function keeps the store type of the source and keeps `Date`/`Date32` columns working, since `addMilliseconds` rejects those outright.
alex-clickhouse and others added 2 commits August 14, 2026 20:56
The provider registered no date/time member translator, so only a direct
comparison worked: every member and method threw "The LINQ expression ...
could not be translated". Add one shared translator, which serves DateTime
and DateOnly because the ClickHouse function is the same for each.

Components map to the to* extraction functions. Those return UInt8/UInt16,
which the provider's integer mappings already widen on read.

.DayOfWeek maps to toDayOfWeek(x, 2). Week mode 2 agrees with
System.DayOfWeek exactly, so no arithmetic correction is applied. The
result carries a number-backed enum mapping, because this provider maps a
C# enum to a ClickHouse string and that mapping would otherwise render
x.DayOfWeek == DayOfWeek.Sunday as a comparison against 'Sunday'.

.TimeOfDay maps to toTime64(x, 7); precision 7 is one .NET tick, so the
fraction survives, which toTime would drop.

AddYears and AddMonths take an int, so they map straight onto addYears and
addMonths, which clamp the day of month the way .NET does.

The other Add* methods take a double, which .NET scales to whole ticks.
The matching ClickHouse function takes a whole number of its own unit and
discards the rest, so addDays(x, 1.5) would add only one day. A constant
is therefore folded to ticks and expressed in the coarsest unit that holds
it exactly: the natural function when possible, otherwise addMilliseconds.
Preferring the natural function keeps the source's store type and keeps
Date/Date32 columns working, which addMilliseconds rejects.

A sub-millisecond offset, a non-constant offset, and a value outside the
DateTime range are left untranslated rather than rounded to fit.
addNanoseconds would express a tick exactly, but promotes the result to
DateTime64(9), whose Int64 nanosecond count cannot span the DateTime64
range — that would trade a rounding error for a silently wrong date.
Server-side rounding was rejected too: ClickHouse round() is banker's
rounding, so it disagrees with .NET. An untranslated call still gives the
correct value through client evaluation in a projection, and reports a
reason in a predicate.

Also report a clear reason for arithmetic on two date/time values. ClickHouse
has no operator that matches the .NET result: one date minus another gives a
TimeSpan while dateDiff counts whole units, Time64 subtraction gives a
Decimal of seconds, and a date plus a TimeSpan is rejected outright. These
used to fail with an internal cast or coercion error naming CLR types the
user never wrote. Reporting the reason also restores client evaluation in a
projection, where the .NET result is correct.

DateTimeOffset follows in the next commit, once the store mapping this
branch is stacked on is in place.

The Northwind GroupJoin_aggregate_anonymous_key_selectors2 query now
translates, so its "not translatable" override is removed.

Co-Authored-By: Claude <noreply@anthropic.com>
The shared translator already takes the CLR type as a parameter, so serving
DateTimeOffset is a registration. It was held back only because the type had
no store mapping: such a property resolved to String, where the extraction
functions fail on the server and addDays silently drops both the offset and
the sub-second part. The mapping from #53, which this branch is stacked on,
removes that obstacle.

The result is in the timezone the column declares, which the store type pins
to UTC. That agrees with .NET, because a value read back from such a column
carries the +00:00 offset, so .Hour and .Date describe the same instant on
both sides.

DateTimeOffset.Now translates to the same UTC-pinned now64 as UtcNow, since a
DateTimeOffset is an instant.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants