BUG: non-monotonic DatetimeIndex does not raise KeyError for missing labels

[shoyer]

import pandas as pd
index = pd.to_datetime(['2000-01-02', '2000-01-01'])
index.get_loc('1900-01-01')

array([], dtype=int64)

[jreback]

this is correct. It cannot tell its out of range because is not monotonic, so its effectively just a mask on the original. I suppose this could raise, what is the usecase?

Scalars raise KeyErrors when not found

In [29]: index.get_loc(Timestamp('2000-1-1'))
Out[29]: 1

In [30]: index.get_loc(Timestamp('1900-1-1'))
KeyError: Timestamp('1900-01-01 00:00:00')

[shoyer]

Honestly in this case it was mostly a mistake -- non-monotonic time indexes are not very useful.

I think the right behavior is probably to raise a KeyError anytime an empty array would be returned from an indexing method. Returning a length 0 array in some cases means it is possible to end up with unexpected empty series and dataframes. Inevitably, you run into an error later (or worse, maybe not!), but it's harder to debug when it is further from the source.

[immerrr]

raise a KeyError anytime an empty array would be returned from an indexing method

Please, don't, it is a legitimate use case to work with empty datasets, e.g. that happens to me when I fill a frame with incoming data over time.

[shoyer]

@immerrr I should clarify: by "indexing method" I guess I just really just meant get_loc (and perhaps get_locs for MultiIndex).

Yes, it's absolutely a legitimate case to work with empty datasets -- but do you expect to be able to index them with scalar labels?

[immerrr]

Ah, ok, sure. I would indeed expect index.get_locs(timestr) and index.get_locs(Timestamp(timestr)) to behave the same, but I guess I just don't know the whole story.

[jreback]

see here: https://github.com/pydata/pandas/blob/master/pandas/tseries/index.py#L1200

I remember putting this in for a monotonic index a while back, where you can easily figure out if its out of range.

but didn't test (obviously) for non-monotonic.

its easy to figure out if its not their, and I am leaning that it should raise (as get_loc/s DOES always raise KeyError if all values are out of range, rather than return an empty).

all that said, I don't know if anything will (or should break).

[jreback]

@shoyer PR for this?

[NumberPiOso]

Nowadays the error is correctly reported

index = pd.to_datetime(['2000-01-02', '2000-01-01'])
index.get_loc('1900-01-01')
File ~/git/mypandas/pandas/core/indexes/datetimes.py:681, in DatetimeIndex.get_loc(self, key, method, tolerance)
    679     return Index.get_loc(self, key, method, tolerance)
    680 except KeyError as err:
--> 681     raise KeyError(orig_key) from err

KeyError: '1900-01-01'

So this issue is solved and should be closed

[jreback]

see if we have a direct test for this (probably) if not would take a PR with one

[NumberPiOso]

Indeed, it was highly probable

pandas/pandas/tests/indexes/datetimes/test_indexing.py

Lines 449 to 453 in aa720d4

	def test_get_loc_reasonable_key_error(self):
	# GH#1062
	index = DatetimeIndex(["1/3/2000"])
	with pytest.raises(KeyError, match="2000"):
	index.get_loc("1/1/2000")