Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: resampling 2D with closed=right fails with "Values falls before first bin" #43198

Open
2 of 3 tasks
batterseapower opened this issue Aug 24, 2021 · 0 comments
Open
2 of 3 tasks
Labels
Bug Resample resample method

Comments

@batterseapower
Copy link
Contributor

batterseapower commented Aug 24, 2021

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of pandas. (1.3.1)
  • (optional) I have confirmed this bug exists on the master branch of pandas.

Code Sample, a copy-pastable example

# ValueError: Values falls before first bin
pd.Series(index=pd.DatetimeIndex(['2013-04-01 22:00:00'])).resample('2D', closed='right').max()

# Works OK
pd.Series(index=pd.DatetimeIndex(['2013-04-01 22:00:00'])).resample('1D', closed='right').max()
pd.Series(index=pd.DatetimeIndex(['2013-04-01 22:00:00'])).resample('2D').max()

(Any number of days > 1 shows the same bug)

Stack trace from failure:

ValueError                                Traceback (most recent call last)
<ipython-input-171-dd7aa6747f09> in <module>
----> 1 itensors.unwrap_1d(pd.Series(np.arange(len(de.times)), index=de.times).resample('14D', closed='right', label='right').max())

~\Anaconda3\envs\py3_2\lib\site-packages\pandas\core\resample.py in f(self, _method, min_count, *args, **kwargs)
    955     def f(self, _method=method, min_count=0, *args, **kwargs):
    956         nv.validate_resampler_func(_method, args, kwargs)
--> 957         return self._downsample(_method, min_count=min_count)
    958 
    959     f.__doc__ = getattr(GroupBy, method).__doc__

~\Anaconda3\envs\py3_2\lib\site-packages\pandas\core\resample.py in _downsample(self, how, **kwargs)
   1051         **kwargs : kw args passed to how function
   1052         """
-> 1053         self._set_binner()
   1054         how = self._get_cython_func(how) or how
   1055         ax = self.ax

~\Anaconda3\envs\py3_2\lib\site-packages\pandas\core\resample.py in _set_binner(self)
    193         """
    194         if self.binner is None:
--> 195             self.binner, self.grouper = self._get_binner()
    196 
    197     def _get_binner(self):

~\Anaconda3\envs\py3_2\lib\site-packages\pandas\core\resample.py in _get_binner(self)
    200         has already been called.
    201         """
--> 202         binner, bins, binlabels = self._get_binner_for_time()
    203         assert len(bins) == len(binlabels)
    204         bin_grouper = BinGrouper(bins, binlabels, indexer=self.groupby.indexer)

~\Anaconda3\envs\py3_2\lib\site-packages\pandas\core\resample.py in _get_binner_for_time(self)
   1040         if self.kind == "period":
   1041             return self.groupby._get_time_period_bins(self.ax)
-> 1042         return self.groupby._get_time_bins(self.ax)
   1043 
   1044     def _downsample(self, how, **kwargs):

~\Anaconda3\envs\py3_2\lib\site-packages\pandas\core\resample.py in _get_time_bins(self, ax)
   1526 
   1527         # general version, knowing nothing about relative frequencies
-> 1528         bins = lib.generate_bins_dt64(
   1529             ax_values, bin_edges, self.closed, hasnans=ax.hasnans
   1530         )

pandas\_libs\lib.pyx in pandas._libs.lib.generate_bins_dt64()

ValueError: Values falls before first bin

Problem description

If nothing falls exactly on the bin edge, closed=right should give exactly the same results as closed=left. However, for this example the closed=right case just fails even though closed=left succeeds.

Expected Output

2013-04-01   NaN
Freq: D, dtype: float64

Output of pd.show_versions()

(pd.show_versions doesn't work on my 1.3.1 install so this is from a different environment.)

INSTALLED VERSIONS

commit : 2cb9652
python : 3.8.10.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.18363
machine : AMD64
processor : Intel64 Family 6 Model 85 Stepping 7, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United Kingdom.1252

pandas : 1.2.4
numpy : 1.20.2
pytz : 2021.1
dateutil : 2.8.1
pip : 21.1.1
setuptools : 52.0.0.post20210125
Cython : 0.29.23
pytest : 6.2.3
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.6.3
html5lib : 1.1
pymysql : None
psycopg2 : None
jinja2 : 3.0.0
IPython : 7.22.0
pandas_datareader: None
bs4 : 4.9.3
bottleneck : 1.3.2
fsspec : 0.9.0
fastparquet : None
gcsfs : None
matplotlib : 3.3.4
numexpr : None
odfpy : None
openpyxl : 3.0.7
pandas_gbq : None
pyarrow : 3.0.0
pyxlsb : None
s3fs : None
scipy : 1.6.2
sqlalchemy : None
tables : None
tabulate : None
xarray : 0.18.0
xlrd : 2.0.1
xlwt : None
numba : 0.53.0

@batterseapower batterseapower added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Aug 24, 2021
@mroeschke mroeschke added Resample resample method and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Sep 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Resample resample method
Projects
None yet
Development

No branches or pull requests

2 participants