chore: audit date/time expressions

[andygrove]

Which issue does this PR close?

Closes #.

Rationale for this change

Only from_utc_timestamp and to_utc_timestamp had per-version audit sub-bullets in spark_expressions_support.md. The rest of the datetime_funcs section was implemented but undocumented as to which Spark versions it had been validated against. This PR brings every implemented [x] date/time function (38 SQL names backed by 33 Comet serde objects) under the same audit format, captures the three correctness divergences found during that audit as ignored regression tests, and applies the support-level / reason consistency fixes the audit surfaced.

What changes are included in this PR?

Documentation

Adds sub-bullets to every [x] entry under ### datetime_funcs in docs/source/contributor-guide/spark_expressions_support.md, except the five entries that have no dedicated Comet serde (current_timezone, date_part, datepart, extract, localtimestamp route through Spark optimizer rewrites to other expressions or evaluate to constants).

For each function, the sub-bullets record:

• Whether the Spark class is identical across 3.4.3, 3.5.8, and 4.0.1.
• Spark 4.0 changes. Universally the NullIntolerant trait migrated to a nullIntolerant: Boolean field override; many string-typed inputTypes widened to StringTypeWithCollation(supportsTrimCollation = true); ANSI error helpers were renamed.
• Known divergences between Comet and Spark, with tracking-issue links where applicable.

The work was driven by 8 parallel agents using the audit-comet-expression skill, each handling a related group of expressions (codegen-dispatched, date field extractors, Hour/Minute/Second, scalar function wrappers, timezone/unix, truncation, format, Iceberg transforms).

Captured regression tests for correctness findings

Three correctness divergences were found and captured as query ignore(<issue-url>) SQL tests. When the underlying bug is fixed, removing the ignore(...) activates the assertion.

Test	Bug	Issue
next_day.sql (appended)	Comet trims ' MO ' to match Monday; Spark does not trim and returns NULL	#4450
next_day_ansi.sql (new)	next_day returns NULL where Spark throws under spark.sql.ansi.enabled=true	#4449
make_date_ansi.sql (new)	make_date returns NULL where Spark throws under spark.sql.ansi.enabled=true	#4451

A fourth audit finding (make_date year 0 / negative years) was checked against Spark's own MakeDate.nullSafeEval, which uses LocalDate.of(year, month, day) and accepts the full Java date range. Spark and chrono agree; the existing make_date.sql tests at lines 133/136 already exercise it. Issue #4452 was closed as not-a-bug, no test added.

Why not fix instead of ignore? Both SparkNextDay and SparkMakeDate live in the upstream datafusion-spark crate (pinned at 53.1.0 in Cargo.lock). Fixing the ANSI throw and the trim behaviour requires changes there, which is outside the scope of this PR.

Support-level consistency fixes

Mechanical fixes for the alignment issues the audit surfaced. No behaviour changes; the only observable effect is that EXPLAIN-time fallback messages now include the specific reason instead of a generic "not fully compatible with Spark":

• New TimeFieldSerde private companion hoists the shared TimestampNTZ reason used by CometHour, CometMinute, CometSecond (mirrors UTCTimestampSerde).
• CometTruncDate: extracted reasons to private vals; corrected "Invalid" vs "Non-literal" wording mismatch.
• CometTruncTimestamp: added the missing non-literal-format reason to getIncompatibleReasons; added missing getUnsupportedReasons override; reasons shared via private vals.
• CometSecondsToTimestamp: added missing getUnsupportedReasons override.
• CometHours / CometDays: added getSupportLevel and getUnsupportedReasons so the unsupported-input-type fallback surfaces in EXPLAIN/Compatibility Guide instead of being silently swallowed via withInfo in convert.
• CometFromUnixTime: moved format-pattern check from convert into getSupportLevel (Unsupported for non-default, Incompatible for default); both reason methods now populated. Updated the existing from_unix_time.sql expect_fallback assertions to match the new, more specific fallback message.

Out of scope here: CometDateFormat (needs semantics decision about gating refactor) and CometUnixTimestamp (predicate-vs-reason disagreement needs verification of what TimestampNTZ actually does). Both are listed in the consistency-findings section below for follow-up.

Remaining consistency findings (follow-up)

These were surfaced by the audit but are out of scope here because they need semantics decisions, not mechanical edits:

• CometUnixTimestamp: getUnsupportedReasons claims TimestampNTZType is not supported, but isSupportedInputType returns true for it. Either the predicate or the reason is wrong; needs verification of what the Rust path actually does with TimestampNTZ.
• CometDateFormat: getSupportLevel returns Compatible() while convert reads allowIncompatible and branches on UTC vs non-UTC. The new skill rule prefers getSupportLevel gating, but moving it here changes the dispatch flow slightly and needs a separate review.
• The 9 codegen-dispatched expressions in Group A (informational): getSupportLevel returns Compatible() while convert returns None when spark.comet.exec.scalaUDF.codegen.enabled=false. The dispatcher flag is not surfaced in the compatibility guide.

How are these changes tested?

Documentation: prettier --check passes on spark_expressions_support.md.

Captured regression tests: the new SQL files parse against SqlFileTestParser (IgnorePattern, ConfigPattern, MinSparkVersionPattern regexes all match). The query ignore(...) queries are skipped at runtime per the documented ignore mode contract in sql-file-tests.md, so they do not affect suite outcomes today and will activate automatically when the upstream datafusion-spark fixes land.

Support-level fixes: ran ./mvnw test -Dsuites="org.apache.comet.CometSqlFileTestSuite datetime/" -Dtest=none locally. All 90 datetime SQL file tests pass.