[C++] Make casting timestamp and duration zero-copy when TimeUnit matches

[icexelloss]

Describe the bug, including details regarding any error messages, version, and platform.

I am testing performance of casting datatypes with pyarrow Table and saw some unexpected performance.

In short, it seems that casting a column from "tz-naive" to "tz-utc" is much slower than casting from "tz-naive" to "int64", which is unexpected because I think both of these should be metadata-only change.

Here is a partial repo:

In [5]: df = pd.DataFrame({'time': np.arange(100 * 1000  * 1000)})

In [6]: table = pa.Table.from_pandas(df)

In [8]: schema_naive = pa.schema([pa.field('time' , pa.timestamp('ns'))])

In [9]: schema_tz = pa.schema([pa.field('time' , pa.timestamp('ns', tz='UTC'))])

In [10]: table = table.cast(schema_naive)

In [14]: schema_int = pa.schema([pa.field('time' , pa.int64()))])

In [16]: %time table.cast(schema_int)
CPU times: user 114 µs, sys: 30 µs, total: 144 µs
Wall time: 231 µs
Out[16]: 
pyarrow.Table
time: int64
----
time: [[0,1,2,3,4,...,99999995,99999996,99999997,99999998,99999999]]

In [17]: %time table.cast(schema_tz)
CPU times: user 119 ms, sys: 140 ms, total: 260 ms
Wall time: 259 ms
Out[17]: 
pyarrow.Table
time: timestamp[ns, tz=UTC]
----
time: [[1970-01-01 00:00:00.000000000,1970-01-01 00:00:00.000000001,1970-01-01 00:00:00.000000002,1970-01-01 00:00:00.000000003,1970-01-01 00:00:00.000000004,...,1970-01-01 00:00:00.099999995,1970-01-01 00:00:00.099999996,1970-01-01 00:00:00.099999997,1970-01-01 00:00:00.099999998,1970-01-01 00:00:00.099999999]]

In [18]: table
Out[18]: 
pyarrow.Table
time: timestamp[ns]
----
time: [[1970-01-01 00:00:00.000000000,1970-01-01 00:00:00.000000001,1970-01-01 00:00:00.000000002,1970-01-01 00:00:00.000000003,1970-01-01 00:00:00.000000004,...,1970-01-01 00:00:00.099999995,1970-01-01 00:00:00.099999996,1970-01-01 00:00:00.099999997,1970-01-01 00:00:00.099999998,1970-01-01 00:00:00.099999999]]

In [19]: pa.__version__
Out[19]: '11.0.0'

Component(s)

Python

[icexelloss]

FWIW I found this comment about the behavior: https://github.com/apache/arrow/blob/master/cpp/src/arrow/compute/kernels/scalar_cast_temporal.cc#L156

[jorisvandenbossche]

Indeed, that seems to be due to not doing this a zero-copy cast because the data types are "different" (while only the unit matters, the timezone can be ignored for this operation).

[rok]

This would would definitely be nice to have.

[icexelloss]

@rok This might be a silly question but why don't we dynamically dispatch to a zero copy / no op functions here if the units are the same?

https://github.com/apache/arrow/blob/master/cpp/src/arrow/compute/kernels/scalar_cast_temporal.cc#L158

[rok]

@icexelloss yeah, no op seems like the thing to do. I wonder if zero copy is possible or does the fact we're working with batches prevent that. @westonpace could you comment?

[westonpace]

I don't think batches should be a problem. It seems we only have the comment from @wesm to go on here. Given that it was made during a rather large refactor my guess is this is more of a "todo" and less of a "this is a concern". I think it should be pretty safe to zero copy (and it is concerning that we don't).

[rok]

Ok, I'll give it a shot.