PERF: array_strptime

[jbrockmendel]

array_strptime is conceptually similar to array_to_datetime but stores local values instead of UTC values, then we pass the result through a localization pass that goes through object dtype. The upside of doing it the current way is that array_strptime does not need a fallback path for when mixed tzs/awareness is detected. The downside is everything else.

This changes array_strptime to store UTC values and follow the same pattern used by array_to_datetime (hopefully we can move toward sharing more code, though some of that will have to wait until deprecations are enforced in 3.0). It introduces a fallback _array_strptime_object_fallback that mirrors _array_to_datetime_object that we'll be able to get rid of once the mixed-tz deprecation is enforced in 3.0.

vals = np.array(["2016-01-01 02:03:04.567"] * 10**4, dtype=object)
vals2 = np.array([pd.Timestamp(x) for x in vals], dtype=object)
vals3 = np.array([x + "-01:00" for x in vals], dtype=object)

fmt = "%Y-%m-%d %H:%M:%S.%f"

%timeit pd.to_datetime(vals, format=fmt, cache=False)
2.39 ms ± 28.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # <- main
2.11 ms ± 45.3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # <- PR

%timeit pd.to_datetime(vals2, format=fmt, cache=False)
23.9 ms ± 680 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)   # <- main
10.7 ms ± 113 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # <- PR


%timeit pd.to_datetime(vals3, format=fmt+"%z", cache=False)
99.7 ms ± 1.43 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <- main
63.1 ms ± 1.65 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <- PR

[WillAyd]

Might need to be careful with performance of this during transition from Cython 0.29 to Cython 3. Definitely if this was just ndarray values you would lose performance if you didn't use PySequence_GetItem in the loop; I'm not sure if declaring it as ndarray[object] values makes a difference

[jbrockmendel]

should i update this to use PySequence_GetItem? im not too concerned with this function since it will be removed in 3.0 anyway

[WillAyd]

If it's going away soon probably not worth the effort

[MarcoGorelli]

Changes look good

There's a lot of duplication between _array_strptime_object_fallback and array_strptime, but as far as I understand, _array_strptime_object_fallback can be removed in 3.0 anyway, right? If so, no bother

Does this need a whatsnew note for the perf improvement? (can be done as part of another PR, feel free to merge)

[jbrockmendel]

but as far as I understand, _array_strptime_object_fallback can be removed in 3.0 anyway, right?

correct.

[mroeschke]

Thanks @jbrockmendel