Resampler sometimes adds extra bin with NaN

[vapoto]

Code Sample

import pandas as pd
idx = pd.timedelta_range(start='8H', freq='10S', periods=5040)
s = pd.Series(range(len(idx)), index=idx)
s.resample('3H').min()

Observed Output

08:00:00       0.0
11:00:00    1080.0
14:00:00    2160.0
17:00:00    3240.0
20:00:00    4320.0
23:00:00       NaN
Freq: 3H, dtype: float64

Expected Output

08:00:00       0.0
11:00:00    1080.0
14:00:00    2160.0
17:00:00    3240.0
20:00:00    4320.0
Freq: 3H, dtype: float64

Problem description

Downsampling and then applying (any) function to the Resampler object seems to add an extra bin in some cases. It is unclear when this exactly occurs, but these two conditions seem necessary (but not sufficient) to reproduce the bug:

• the resampling period does not exactly fit in the original index
• the original index starts with a non-zero Timedelta

This guess is based on the fact that changing '8H' to '0H' in the code above returns an expected output. Equally so does changing the resampling period to '2H'.

Output of pd.show_versions()

INSTALLED VERSIONS

commit : None
python : 3.7.5.final.0
python-bits : 64
OS : Linux
OS-release : 5.0.0-37-generic
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_GB.UTF-8
LOCALE : en_GB.UTF-8

pandas : 0.25.3
numpy : 1.17.4
pytz : 2019.3
dateutil : 2.8.1
pip : 19.3.1
setuptools : 42.0.2.post20191203
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.10.3
IPython : 7.10.2
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : 3.1.1
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
s3fs : None
scipy : 1.3.2
sqlalchemy : None
tables : None
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None

[hasB4K]

take

[hasB4K]

Hello @vapoto 🙂,

FYI, I have tracked the error in #33498, it came from a bug of pd.timedelta_ranges (a bug that used to exists for pd.date_range, but it seems it was not implemented for timeldetas).

Your example gives the proper output on my PR.

On master without the fix:

0 days 08:00:00       0.0
0 days 11:00:00    1080.0
0 days 14:00:00    2160.0
0 days 17:00:00    3240.0
0 days 20:00:00    4320.0
0 days 23:00:00       NaN
Freq: 3H, dtype: float64

On my PR:

0 days 08:00:00       0
0 days 11:00:00    1080
0 days 14:00:00    2160
0 days 17:00:00    3240
0 days 20:00:00    4320
Freq: 3H, dtype: int64

Hopefully this should solve your issue!

[vapoto]

Hi @hasB4K,

Thanks a lot for that! Seems like that would indeed solve the problem. I just added a suggestion to your PR to add tests for some of the cases that are currently working.