[feature](timestamp_ns) Add basic TIMESTAMP_NS type support - #66702
Open
jacktengg wants to merge 1 commit into
Open
[feature](timestamp_ns) Add basic TIMESTAMP_NS type support#66702jacktengg wants to merge 1 commit into
jacktengg wants to merge 1 commit into
Conversation
### What problem does this PR solve? Issue Number: None Related PR: apache#66333 Problem Summary: Doris DATETIME and DATETIMEV2 support at most six fractional-second digits. Extending their existing packed civil-time representation to nanoseconds would mix incompatible storage representations and semantics. Introduce TIMESTAMP_NS as a separate, fixed-precision timestamp type: - TIMESTAMP_NS always has nine fractional-second digits and does not accept a precision parameter. - Values are stored as signed Int64 nanoseconds relative to the Unix epoch. - The supported range is [1677-09-21 00:12:43.145224192, 2262-04-11 23:47:16.854775807]. - TIMESTAMP_NS has no time-zone attribute. UTC is used as the stable mapping between its civil-time fields and physical epoch-nanosecond representation. - DATETIME(p) and DATETIMEV2(p) retain their existing behavior and continue to support only precision 0 through 6. Add the new primitive type to Thrift, protobuf, FE type metadata, Nereids, BE data types, and cloud schema metadata. Add dedicated TIMESTAMP_NS literal, value, column, and SerDe implementations. Support the basic end-to-end SQL and storage paths, including: - DDL, literals, default values, partitions, buckets, and schema metadata - Insert, stream load, partial update, and decoded timestamp input - Segment storage, key coding, zone map, bloom filter, and inverted index - Comparison, predicates, hashing, grouping, ordering, joins, runtime filters, and basic aggregate functions - FE literal validation, constant folding, type coercion, and predicate simplification - MySQL text and prepared-statement binary results, using string-compatible output to preserve all nine fractional digits Arrow Flight SQL output, ORC OUTFILE, and Java UDF signatures explicitly reject TIMESTAMP_NS because those integrations are not supported yet. The broader explicit cast matrix and scalar date/time function signatures will be implemented in follow-up changes. ### Release note Add the fixed-precision TIMESTAMP_NS SQL type for nanosecond timestamps. TIMESTAMP_NS stores signed Int64 Unix-epoch nanoseconds, supports the range from 1677-09-21 00:12:43.145224192 through 2262-04-11 23:47:16.854775807, and always displays nine fractional digits. Existing DATETIME and DATETIMEV2 types remain limited to precision 0 through 6. ### Check List (For Author) - Test: Unit Test and Regression Test - Added BE unit tests for TIMESTAMP_NS values, SerDe, storage indexes, hashing, predicates, aggregates, and partitioning - Added FE unit tests for types, literals, boundaries, partitions, coercion, folding, MySQL output, and unsupported integrations - Added regression tests for DDL, predicates, storage, stream load, partial/default values, and MySQL prepared-statement output - Behavior changed: Yes; adds the TIMESTAMP_NS SQL and storage type without changing DATETIME or DATETIMEV2 precision semantics - Does this need documentation: Yes; TIMESTAMP_NS syntax, range, storage semantics, and current integration limitations require documentation
jacktengg
requested review from
924060929,
airborne12,
csun5285,
eldenmoon,
englefly,
gavinchou,
liaoxin01,
luwei16,
morrySnow,
mymeiyi,
seawinde,
starocean999 and
yiguolei
as code owners
August 12, 2026 14:15
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: None
Problem Summary:
Doris DATETIME and DATETIMEV2 support at most six fractional-second
digits. Extending their existing packed civil-time representation to
nanoseconds would mix incompatible storage representations and semantics.
Introduce TIMESTAMP_NS as a separate, fixed-precision timestamp type:
Add the new primitive type to Thrift, protobuf, FE type metadata, Nereids,
BE data types, and cloud schema metadata. Add dedicated TIMESTAMP_NS literal,
value, column, and SerDe implementations.
Support the basic end-to-end SQL and storage paths, including:
Arrow Flight SQL output, ORC OUTFILE, and Java UDF signatures explicitly
reject TIMESTAMP_NS because those integrations are not supported yet.
The broader explicit cast matrix and scalar date/time function signatures
will be implemented in follow-up changes.
Release note
Add the fixed-precision TIMESTAMP_NS SQL type for nanosecond timestamps.
TIMESTAMP_NS stores signed Int64 Unix-epoch nanoseconds, supports the range
from 1677-09-21 00:12:43.145224192 through
2262-04-11 23:47:16.854775807, and always displays nine fractional digits.
Existing DATETIME and DATETIMEV2 types remain limited to precision 0 through 6.
Check List (For Author)
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)