[C++/Python] Create Array from sequence of numpy.datetime64

[asfimport]

Currently we only support datetime.datetime and datetime.date but numpy.datetime64 also occurs quite often in the numpy/pandas-related world.

Reporter: Uwe Korn / @xhochy
Assignee: Krisztian Szucs / @kszucs

PRs and other links:

• GitHub Pull Request #2528

_{Note: This issue was originally created as ARROW-1963. Please see the migration documentation for further details.}

[asfimport]

Antoine Pitrou / @pitrou:
Can you qualify what is desired here? Currently we have the following:

>>> a = np.array(['2001', '2002', '2003'], dtype='M8[ms]')
>>> a
array(['2001-01-01T00:00:00.000', '2002-01-01T00:00:00.000',
       '2003-01-01T00:00:00.000'], dtype='datetime64[ms]')
>>> pa.array(a)
<pyarrow.lib.TimestampArray object at 0x7f72e90ad5e8>
[
  Timestamp('2001-01-01 00:00:00'),
  Timestamp('2002-01-01 00:00:00'),
  Timestamp('2003-01-01 00:00:00')
]

and also:

>>> pa.array(list(a), type=pa.timestamp('ms'))
<pyarrow.lib.TimestampArray object at 0x7f72e9141818>
[
  Timestamp('2001-01-01 00:00:00'),
  Timestamp('2002-01-01 00:00:00'),
  Timestamp('2003-01-01 00:00:00')
]

but not:

>>> pa.array(list(a))
Traceback (most recent call last):
  File "<ipython-input-70-bca21b085475>", line 1, in <module>
    pa.array(list(a))
  File "array.pxi", line 181, in pyarrow.lib.array
  File "array.pxi", line 26, in pyarrow.lib._sequence_to_array
  File "error.pxi", line 85, in pyarrow.lib.check_status
ArrowNotImplementedError: /home/antoine/arrow/cpp/src/arrow/python/builtin_convert.cc:983 code: AppendPySequence(seq, size, real_type, builder.get())
/home/antoine/arrow/cpp/src/arrow/python/builtin_convert.cc:406 code: static_cast<Derived*>(this)->AppendSingle(ref.obj())
Cannot convert NumPy datetime64 objects with differing unit

[asfimport]

Wes McKinney / @wesm:
Yes, I think we would want the last example to behave like the others. This could assume that all of the datetime64 scalar values' units are the same (and raise exception if not)

[asfimport]

Uwe Korn / @xhochy:
Issue resolved by pull request 2528
#2528