API: to_datetime inconsistent returning of datetime.datetime

[jorisvandenbossche]

Copying from #14448 (comment) (as the issue is closed now):

In [1]: pd.to_datetime('13000101', errors='ignore')
Out[1]: '13000101'

In [2]: pd.to_datetime('13000101', errors='ignore', format='%Y%m%d')
Out[2]: datetime.datetime(1300, 1, 1, 0, 0)

The above inconsistency (which is also not documented at all I think), is that something we want to fix?

[jreback]

i would say we should return the same as [1]
though is it then not possible to get a datetime for out of bounds dates?

[tacaswell]

maybe bread crumbs back to #6415

[jorisvandenbossche]

It also depends on which format is specified:

In [38]: pd.to_datetime(['15010101', '20150101'], format="%Y%m%d", errors='ignore')
Out[38]: Index([1501-01-01 00:00:00, 2015-01-01 00:00:00], dtype='object')  <-- datetime.datetime objects

In [40]: pd.to_datetime(['1501-01-01', '2015-01-01'], format="%Y-%m-%d", errors='ignore')
Out[40]: Index(['1501-01-01', '2015-01-01'], dtype='object')

[jorisvandenbossche]

On the original question of which of the two behaviours we would want: whether it is for errors='ignore' or not, I think there is some value in having the option to get datetime.datetime values for out of bound dates

[nathalier]

Yet another example of unpredictable behaviour:

pd.to_datetime(12912, format="%Y%m%d", errors='ignore')
Out[4]: 12912
pd.to_datetime(21213, format="%Y%m%d", errors='ignore')
Out[5]: datetime.datetime(2, 12, 13, 0, 0)

[MarcoGorelli]

yeah it's the %Y%m%d fastpath and its myriad of bugs

this'll be addressed in #50242

[jorisvandenbossche]

Resolved by #50242