Datetime support

[teoliphant]

This library needs standard date, time, and datetime support. NumPy's model should be use (with the adjustment that parsing dates and times where no time-zone is specified should provide a time-zone independent datetime with an implicit date-time assumed by the user of the data).

[dhirschfeld]

I think I agree with the above statement but it reads slightly ambiguously for me.

The current numpy model is a reasonable model though mostly sidesteps timezone issues . I just want to make it clear that the original numpy model of implicitly assuming whatever timezone the user was in was very bad and shouldn't be replicated!

xref: numpy/numpy#3388, numpy/numpy#6453

[skrah]

This is a small digression from the actual issue, but since there have been questions about the extensibility of ndtypes and xnd, here is how it would work for datetime without recompiling:

Use the constructor syntax to mark an int64_t as Datetime:

>>> x = xnd.empty("3 * Datetime(int64)")
>>> x.type
ndt("3 * Datetime(int64)")
>>> x.value
[0, 0, 0]

If another application wants to use e.g. tai64na (https://cr.yp.to/libtai/tai64.html), which needs 16 uint8_t, use the constructor syntax to create a named array dtype:

>>> x = xnd.empty("3 * Tai64NA(16 * uint8)")
>>> x.type
ndt("3 * Tai64NA(16 * uint8)")
>>> x.value
[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]

Pattern matching will distinguish the constructed types and prevent gufuncs from using the wrong one (e.g. in the case on tai64na an untagged array of 16 uint8_t).

[skrah]

Back to the original issue. If we add datetime as a first class type, ndtypes only cares about the type name, the alignment and the size.

I understand from NumPy that this is always an int64_t. We could do something analogous to char('utf8') and use datetime64('as'), datetime64('Y') etc.

But that is only a small part of the support. Interpreting the data should be part of xnd -- the datetime code in NumPy looks quite large, something like 3000 lines of C. I hope we can re-use some of that.

[deepankarsharma]

What do you think of the idea about splitting datetime/time and timezone into two orthogonal concepts. Having timezones living inside a field within a datetime / time object never sat well with me. On many workloads all time related objects live within the same timezone and hence would benefit from the timezone being stored at a much higher level of abstraction than on individual datetime objects.