Skip to content

Allow comparison of Timestamps with different Timezones #11653

@jeffreyssmith2nd

Description

@jeffreyssmith2nd

Is your feature request related to a problem or challenge?

Currently, comparing two times with different timezones gives an error (see reproducer below). The error occurs when attempting type coercion on the two inputs to the less greater than comparison.

You can see the reproducer below, this feels like a natural query that someone may want to execute (they want the data back in UTC, but know the time based on their local timezone for filtering).

Describe the solution you'd like

According to the arrow documentation, all timestamps with timezones are stored relative to UTC epoch, with the timezone used to offset. In theory this should allow us to compare two times from different timezones.

The question becomes what type should the comparison return? Always the type on the left, the right, UTC if exists, something else? In this specific example, we know that UTC would be an appropriate Signature type based on the rest of the SELECT statement, maybe there is enough information (or enough can be provided) to make that determination during planning.

Alternatively, Signature::comparison could be updated to take separate types for the left and right hand sides.

Describe alternatives you've considered

No response

Additional context

Simple Reproducer

copy (
 values
  (arrow_cast('2024-07-25', 'Timestamp(Nanosecond, Some("UTC"))')),
  (arrow_cast('2024-07-24', 'Timestamp(Nanosecond, Some("UTC"))'))
)
to 'example.parquet';
select column1 from 'example.parquet' where column1 > '2020-06-01T00:00:00' at time zone 'America/New_York';

Provides the error:

Error during planning: Cannot infer common argument type for comparison operation Timestamp(Nanosecond, Some("UTC")) > Timestamp(Nanosecond, Some("America/New_York"))

Metadata

Metadata

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions