Change input for to_timestamp function to be seconds rather than nanoseconds, add to_timestamp_nanos

[comphead]

Which issue does this PR close?

Closes #7802.
Closes #2979

Rationale for this change

to_timestamp() builtin function had an input in nanoseconds whereas other SQL engines has input in seconds and as the result the function output gives unexpected result.

What changes are included in this PR?

Changed signature to to_timestamp function from nanoseconds to seconds, also introduced to_timestamp_nano to create timestamp value from nanoseconds

Are these changes tested?

Yes

Are there any user-facing changes?

The user might expect that to_timestamp function will return other values if the input still in nanoseconds

[Weijun-H]

LGTM! Thanks @comphead

[alamb]

I think this makes sense and aligns DataFusion with postgres. Thank you @comphead and @Weijun-H

https://www.postgresql.org/docs/current/functions-datetime.html#FUNCTIONS-DATETIME-TABLE

[alamb]

maybe we can adjust the literal so it still evaluates to true so that we don't lose coverage 🤔

[alamb]

cc @gruuya @Dandandan and @waitingkuo and @waynexia as other people I know who might be using the timestamp functionality of DataFusion and be affected by this change

[mhilton]

Is this documentation still accurate? Integers are now parsed as "unix second timestamps"?. Unless they aren't in which case a little bit more expanation of the behaviour for integers would be helpful.

[waitingkuo]

Thank you @comphead @alamb

This looks great, this closes #2979 as well.

To align with Postgrseql, perhaps we could fire another ticket for supporting more inputs like floating number & string.
i.e. in postgresql

willy=# select to_timestamp('1.1');
       to_timestamp       
--------------------------
 1970-01-01 08:00:01.1+08
(1 row)

willy=# select to_timestamp(1.1);
       to_timestamp       
--------------------------
 1970-01-01 08:00:01.1+08

[alamb]

To align with Postgrseql, perhaps we could fire another ticket for supporting more inputs like floating number & string.

Great idea @waitingkuo. This is now tracked by two tickets:

• Support float arguments to to_timestamp function #7868
• Support to_timestamp with 2 arguments (timestamp format) #5398

[alamb]

while writing up #7868 it occured to me this signature prevents implementing something like to_timestamp(1.234) reasonably, as it can't represent sub-second precison

What would you think about leaving the return type the same (Timestamp(Nanosecond, None)) so that we can add support for floating point argments as a follow on?

[waitingkuo]

Return Timestamp(Nanosecond, None) seems reasonable to me since most of the time related functions in datafusion return nanosecond based timestamp i.e. timestamp '2000-01-01T00:00:00', now(), ...

[comphead]

In PG

select pg_typeof(to_timestamp(1.234))

the return type is timestamp with time zone which has minimum resolution as a microsecond

https://www.postgresql.org/docs/current/datatype-datetime.html

In light of this, having the return type as nanos or micros makes more sense for me.
What do you think is better? microseconds like in PG or nanos like it was in DF before the PR?

[alamb]

I don't have any preference now that we have coercion rules that can automatically coerce between timestamp resolution

[comphead]

got it, I'll do micros as a return type as it is in PG, will update the PR soon

[comphead]

set return type to nanos as it was before.

[waitingkuo]

it still returns Timestamp(second, None) while the input is Int64

❯ select arrow_typeof(to_timestamp(0));
+--------------------------------------+
| arrow_typeof(to_timestamp(Int64(0))) |
+--------------------------------------+
| Timestamp(Second, None)              |
+--------------------------------------+
1 row in set. Query took 0.003 seconds.

[waitingkuo]

@alamb @comphead
any idea why the function could return Timestamp(Second, None) even though we set the return type here as Timestamp(Nanosecond, None)?

[comphead]

Thanks @waitingkuo for raising it.
I suppose for bunch of functions the return type gets changed in create_physical_expr because functions.rs is on physical expr layer and rewrites the return type defined on logical layer.

For ints I would think the second is probably better return type
If you try to present 0004-01-01 year in nanoseconds it will require higher value than i64::MAX/MIN
https://doc.rust-lang.org/src/core/num/mod.rs.html#366

make_type!(
    TimestampNanosecondType,
    i64,
    DataType::Timestamp(TimeUnit::Nanosecond, None),
    "A timestamp nanosecond type with an optional timezone."
);

But you are right we need to clean up builtin functions configuration in multiple places.

[Dandandan]

We could map to_timestamp to ToTimestampSeconds as well durig planing.

[comphead]

@alamb should we merge the PR?

[alamb]

I think so @comphead -- it has had plenty of review by now and no objections.