TimeGrouper only works properly when using a sorted DateTimeIndex

[aschilling]

related SO question

http://stackoverflow.com/questions/19636370/dataframe-groupbytimegrouper-d-invalid-length-for-values-or-for-binner/19636874?noredirect=1#comment29155715_19636874

Hi everybody,

today I spend the whole afternoon figuring out that TimeGrouper only works properly when using on a sorted(!) DateTimeIndex.

If the DateTimeIndex is not sorted, TimeGrouper throws no error (!) but produces corrupt results.

I would suggest either to do a sort() call within TimeGrouper or modify it to throw an error message. The fact that it silently produces corrupt results when the DateTimeIndex is not sorted is really disturbing.

Sorry, that I cannot add an example for the bug. All simple examples I created worked fine, it seems that this only leads to corrupt results with DataFrames of size 700 and above.

Best regards

Andy

[jreback]

can put up an example? (if you can repro)

[aschilling]

Sorry for the delay, I had to wait until the problem reoccured. Take the following DataFrame:

import datetime as DT
df = pd.DataFrame({
'Buyer': 'Carl Carl Carl Carl Joe Carl'.split(),
'Quantity': [18,3,5,1,9,3],
'Date' : [
DT.datetime(2013,9,1,13,0),
DT.datetime(2013,9,1,13,5),
DT.datetime(2013,10,1,20,0),
DT.datetime(2013,10,3,10,0),
DT.datetime(2013,12,2,12,0),
DT.datetime(2013,9,2,14,0),
]})
df = df.set_index(['Date'])

I you execute

df.groupby(TimeGrouper(freq='5D')).sum()

you get different results, based on wether you do

df.sort_index(inplace=True)

before the groupby or not.

Best regards

Andy

[jreback]

I think its requierd to be sorted, so should just raise ...will look in 0.14...(unless you want to do a PR soon!)