Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: "ValueError: Values falls after last bin" on resample('1D') of df with DST index, new in 0.15.2 #9173

Closed
fantabolous opened this issue Dec 30, 2014 · 3 comments
Labels
Bug Timezones Timezone data dtype
Milestone

Comments

@fantabolous
Copy link

I just upgraded to pandas 0.15.2 and now hit this when I do resample. It worked fine in 0.15.1 (I rolled back to 0.15.1 to confirm). As far as I can tell it occurs when the first date in the index is during daylight savings and the last is not, or vice versa.

For example, with a date just before and another just after the DST change date of April 6:

df = pd.DataFrame(index=pd.DatetimeIndex(['2014-04-04 10:00:00', '2014-04-07 10:00:00']), data={'open':0.5}).tz_localize('Australia/Sydney')
print(df.resample('1D', how={'open':'first'}))

Traceback (most recent call last):
File "D:/Documents/Programming/Projects/IB/play.py", line 1332, in
main()
File "D:/Documents/Programming/Projects/IB/play.py", line 48, in main
print(df.resample('1D', how={'open':'first'}))
File "C:\Users\Tom\Anaconda\envs\py2test\lib\site-packages\pandas\core\generic.py", line 3005, in resample
return sampler.resample(self).finalize(self)
File "C:\Users\Tom\Anaconda\envs\py2test\lib\site-packages\pandas\tseries\resample.py", line 85, in resample
rs = self._resample_timestamps()
File "C:\Users\Tom\Anaconda\envs\py2test\lib\site-packages\pandas\tseries\resample.py", line 275, in _resample_timestamps
self._get_binner_for_resample(kind=kind)
File "C:\Users\Tom\Anaconda\envs\py2test\lib\site-packages\pandas\tseries\resample.py", line 123, in _get_binner_for_resample
self.binner, bins, binlabels = self._get_time_bins(ax)
File "C:\Users\Tom\Anaconda\envs\py2test\lib\site-packages\pandas\tseries\resample.py", line 184, in _get_time_bins
bins = lib.generate_bins_dt64(ax_values, bin_edges, self.closed, hasnans=ax.hasnans)
File "pandas\lib.pyx", line 1064, in pandas.lib.generate_bins_dt64 (pandas\lib.c:17790)
ValueError: Values falls after last bin

On the other hand if I roll back to 0.15.1 I get what I expect:
open
2014-04-04 00:00:00+11:00 0.5
2014-04-05 00:00:00+11:00 NaN
2014-04-06 00:00:00+11:00 NaN
2014-04-07 00:00:00+10:00 0.5

Alternatively in 0.15.2 if I add another date in the previous year, such that the first and last dates are either both in daylight savings (even in a different year) or both not in daylight savings, it also seems to work fine.

@fantabolous
Copy link
Author

INSTALLED VERSIONS

commit: None
python: 2.7.6.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 42 Stepping 7, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.15.2
nose: 1.3.4
Cython: 0.21
numpy: 1.9.1
scipy: 0.14.0
statsmodels: 0.5.0
IPython: 2.2.0
sphinx: 1.2.3
patsy: 0.3.0
dateutil: 1.5
pytz: 2014.9
bottleneck: None
tables: 3.1.1
numexpr: 2.3.1
matplotlib: 1.4.2
openpyxl: 1.8.5
xlrd: 0.9.3
xlwt: 0.7.5
xlsxwriter: 0.5.7
lxml: 3.3.5
bs4: 4.3.2
html5lib: 0.999
httplib2: None
apiclient: None
rpy2: None
sqlalchemy: 0.9.7
pymysql: None
psycopg2: None

@fantabolous fantabolous changed the title BUG: "ValueError: Values falls after last bin" on resample('1D') of df with DST index BUG: "ValueError: Values falls after last bin" on resample('1D') of df with DST index, new in 0.15.2 Dec 30, 2014
@jreback jreback added Bug Timezones Timezone data dtype labels Jan 2, 2015
@jreback jreback added this to the 0.16.0 milestone Jan 2, 2015
@jreback
Copy link
Contributor

jreback commented Jan 2, 2015

@fantabolous adding to the master DST resample issues. Which I believe

cc @rockg is tackling

pull-requests welcome of course.

@jreback jreback modified the milestones: 0.16.0, Next Major Release Mar 6, 2015
@jreback jreback modified the milestones: 0.16.0, Next Major Release Mar 10, 2015
@jreback
Copy link
Contributor

jreback commented Mar 11, 2015

closed by #9623

@jreback jreback closed this as completed Mar 11, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Timezones Timezone data dtype
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants