[SPARK-57460][SQL] Support nanosecond-precision timestamp types in the JDBC datasource

[stevomitric]

What changes were proposed in this pull request?

Adds read and write support for the nanosecond-capable timestamp types TIMESTAMP_NTZ(p) / TIMESTAMP_LTZ(p) (p in 7-9) to the built-in JDBC datasource, reaching parity with the microsecond TimestampType / TimestampNTZType, and removes the SPARK-57166 JDBC write rejection guardrail.

Specifically:

• JdbcUtils.getCommonJDBCType: maps the nanos types to SQL TIMESTAMP(p) (dialects may override, e.g. to emit TIMESTAMP(p) WITH TIME ZONE).
• JdbcUtils.getCatalystType / getSchema: a new preferTimestampNanos JDBC option maps a driver TIMESTAMP reporting a sub-microsecond scale (7-9) to a nanos type, gated by the spark.sql.timestampNanosTypes.enabled preview flag. The default microsecond mapping is preserved.
• JDBCValueGetter: TimestampNTZNanosGetter reads the wall-clock directly via getObject(LocalDateTime) (mirroring TimeGetter, avoiding a zone shift); TimestampLTZNanosGetter mirrors the micro TimestampGetter and re-attaches the sub-microsecond digits from java.sql.Timestamp.getNanos. Both floor to the column precision.
• JdbcUtils.makeSetter: NTZ writes the LocalDateTime wall-clock via setObject; LTZ writes the micro java.sql.Timestamp and restores full nanos.
• CreatableRelationProvider.supportsDataType (interfaces.scala): whitelists the nanos types so JDBC write is no longer rejected.

Why are the changes needed?

JDBC was the last built-in file/serde datasource still rejecting nanosecond timestamp types (umbrella SPARK-56822). This extends the feature to JDBC.

Does this PR introduce any user-facing change?

Yes, only for the preview feature gated by spark.sql.timestampNanosTypes.enabled (disabled by default). Nanos timestamp columns can now be written to JDBC, and a driver TIMESTAMP can be read as a nanos type via option("preferTimestampNanos", "true").

How was this patch tested?

Added read (NTZ/LTZ, default-micros-preserved) and write round-trip (precisions 7-9, NTZ and LTZ) tests against H2 in JDBCSuite / JDBCWriteSuite, replacing the SPARK-57166 rejection test. All JDBC suites pass.

Was this patch authored or co-authored using generative AI tooling?

Co-authored-by: Claude Code (Claude Opus 4.8)

[stevomitric]

cc @uros-b , @cloud-fan PTAL when you get a chance.

[uros-b]

Thank you @stevomitric, mostly looks good

[cloud-fan]

1 blocking, 1 non-blocking, 0 nits.
The core conversion paths are coherent, but the new option is incomplete for dialect overrides and is not yet documented for users.

Already raised in existing discussion (1)

• Expose preferTimestampNanos in column metadata before calling JdbcDialect.getCatalystType, matching isTimestampNTZ, so dialect overrides can honor the user option. -- existing discussion

Suggestions (1)

• sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala:379: Document preferTimestampNanos, its default, and its dependency on the nanos preview flag in the JDBC options table. -- see inline

Verification

I traced the option from JDBCOptions through both schema-fetch entry points into JdbcUtils.getSchema, then followed inferred nanos types through getter selection and the NTZ/LTZ setters. The default path still returns the microsecond type, and the new tests cover both opt-in variants and precisions 7-9. The custom-dialect path runs before the fallback and receives metadata containing isTimestampNTZ but not preferTimestampNanos, confirming the existing reviewer concern.

[cloud-fan]

2 addressed, 0 remaining, 1 new to this AI review. (0 newly introduced, 1 late catch, 0 previously raised.)
0 blocking, 1 non-blocking, 0 nits.
The conversion and integration paths are coherent; one small per-column efficiency improvement remains.

Suggestions (1)

• sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala:251: Check the reported scale before reading conf.timestampNanosTypesEnabled. getCatalystType runs once per JDBC column, so the current order performs an active SQLConf lookup for every ordinary TIMESTAMP(0-6) whenever preferTimestampNanos is enabled, even though those scales cannot select a nanos type. Putting the two scale bounds before the config check preserves the result and short-circuits that repeated work. -- see inline

Verification

I traced both schema-fetch entry points through dialect metadata and fallback inference, then followed the inferred NTZ/LTZ nanos types through getter and setter selection. The default path remains microsecond-based, the opt-in path is bounded to scales 7-9, and the changed tests cover default inference plus NTZ/LTZ round trips at precisions 7-9. I did not run the test suites.

[cloud-fan]

Check the scale bounds before reading conf.timestampNanosTypesEnabled. This method runs once per JDBC column, so the current order performs an active SQLConf lookup for every TIMESTAMP(0-6) whenever the option is enabled, even though those scales cannot select a nanos type. Reordering the conjunction preserves behavior and short-circuits that repeated work.