New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

COMPAT: datetime.date comparisons with datetime64[ns] #17965

Closed
jreback opened this Issue Oct 24, 2017 · 9 comments

Comments

Projects
None yet
5 participants
@jreback
Contributor

jreback commented Oct 24, 2017

This should probably work

In [16]: df = pd.DataFrame({'A':[pd.Timestamp('20130101'), np.nan]})

In [17]: df
Out[17]: 
           A
0 2013-01-01
1        NaT

In [18]: pd.Timestamp('20130101').to_pydatetime()>=df.A
Out[18]: 
0     True
1    False
Name: A, dtype: bool

In [20]: pd.Timestamp('20130101').to_pydatetime().date()>=df.A
ValueError: cannot set a Timestamp with a non-timestamp
@jbrockmendel

This comment has been minimized.

Show comment
Hide comment
@jbrockmendel

jbrockmendel Oct 26, 2017

Member

What would date(2017, 10, 24) < Timestamp('2017-10-24 17:02') be?

Member

jbrockmendel commented Oct 26, 2017

What would date(2017, 10, 24) < Timestamp('2017-10-24 17:02') be?

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Oct 26, 2017

Contributor

These just need wrapping

In [7]: pd.Timestamp(datetime.date(2017, 10, 24)) < pd.Timestamp('2017-10-24 17:02')
Out[7]: True
Contributor

jreback commented Oct 26, 2017

These just need wrapping

In [7]: pd.Timestamp(datetime.date(2017, 10, 24)) < pd.Timestamp('2017-10-24 17:02')
Out[7]: True
@jbrockmendel

This comment has been minimized.

Show comment
Hide comment
@jbrockmendel

jbrockmendel Oct 26, 2017

Member

I think the internally consistent thing to do would be to wrap date(2017, 10, 24) with pd.Period(date(2017, 10, 24), 'D')

(the constructor should also be able to infer the freq should be "D", but that is a different issue)

Member

jbrockmendel commented Oct 26, 2017

I think the internally consistent thing to do would be to wrap date(2017, 10, 24) with pd.Period(date(2017, 10, 24), 'D')

(the constructor should also be able to infer the freq should be "D", but that is a different issue)

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Oct 26, 2017

Contributor

no it’s a Timestamp (conceptually it’s just a shortened datetime)

Contributor

jreback commented Oct 26, 2017

no it’s a Timestamp (conceptually it’s just a shortened datetime)

@jbrockmendel

This comment has been minimized.

Show comment
Hide comment
@jbrockmendel

jbrockmendel Oct 26, 2017

Member

(conceptually it’s just a shortened datetime)

That part isn't obvious to me, but conditional on this, then I agree this issue is straightforward.

Member

jbrockmendel commented Oct 26, 2017

(conceptually it’s just a shortened datetime)

That part isn't obvious to me, but conditional on this, then I agree this issue is straightforward.

@TomAugspurger

This comment has been minimized.

Show comment
Hide comment
@TomAugspurger

TomAugspurger Oct 26, 2017

Contributor

FWIW, python raises here

In [4]: datetime.datetime(2017, 1, 1, 12) > datetime.date(2017, 1, 1)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-4-c0cf43ef6360> in <module>()
----> 1 datetime.datetime(2017, 1, 1, 12) > datetime.date(2017, 1, 1)

TypeError: can't compare datetime.datetime to datetime.date
Contributor

TomAugspurger commented Oct 26, 2017

FWIW, python raises here

In [4]: datetime.datetime(2017, 1, 1, 12) > datetime.date(2017, 1, 1)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-4-c0cf43ef6360> in <module>()
----> 1 datetime.datetime(2017, 1, 1, 12) > datetime.date(2017, 1, 1)

TypeError: can't compare datetime.datetime to datetime.date
@liuli1735

This comment has been minimized.

Show comment
Hide comment
@liuli1735

liuli1735 Nov 8, 2017

dates = data['TradingDay'].unique()
for date in dates:
print(date)
raw_data = data.loc[data['TradingDay'] == date].copy()

ValueError: cannot set a Timestamp with a non-timestamp```

I get the same error, i just upgrade my Anaconda yesterday.

liuli1735 commented Nov 8, 2017

dates = data['TradingDay'].unique()
for date in dates:
print(date)
raw_data = data.loc[data['TradingDay'] == date].copy()

ValueError: cannot set a Timestamp with a non-timestamp```

I get the same error, i just upgrade my Anaconda yesterday.
@TomAugspurger

This comment has been minimized.

Show comment
Hide comment
@TomAugspurger

TomAugspurger Nov 8, 2017

Contributor
Contributor

TomAugspurger commented Nov 8, 2017

@akrherz

This comment has been minimized.

Show comment
Hide comment
@akrherz

akrherz Nov 8, 2017

I think a MCVE for @liuli1735 's post is this

import pandas as pd
df = pd.DataFrame({'dates': pd.date_range('2000-01-01', '2010-12-31')})
date = df['dates'].unique()[0]
print(df.loc[df['dates'] == date])

Result with 0.20.3-py27_1 on conda-forge (linux64 python2.7)

$ python mcve.py 
       dates
0 2000-01-01

Result with 0.21.0-py27_0 on conda-forge (linux64 python2.7)

$ python mcve.py 
Traceback (most recent call last):
  File "mcve.py", line 5, in <module>
    print(df.loc[df['dates'] == date])
  File "/opt/miniconda2/lib/python2.7/site-packages/pandas/core/ops.py", line 879, in wrapper
    res = na_op(values, other)
  File "/opt/miniconda2/lib/python2.7/site-packages/pandas/core/ops.py", line 808, in na_op
    y = libindex.convert_scalar(x, _values_from_object(y))
  File "pandas/_libs/index.pyx", line 494, in pandas._libs.index.convert_scalar
  File "pandas/_libs/index.pyx", line 509, in pandas._libs.index.convert_scalar
ValueError: cannot set a Timestamp with a non-timestamp

akrherz commented Nov 8, 2017

I think a MCVE for @liuli1735 's post is this

import pandas as pd
df = pd.DataFrame({'dates': pd.date_range('2000-01-01', '2010-12-31')})
date = df['dates'].unique()[0]
print(df.loc[df['dates'] == date])

Result with 0.20.3-py27_1 on conda-forge (linux64 python2.7)

$ python mcve.py 
       dates
0 2000-01-01

Result with 0.21.0-py27_0 on conda-forge (linux64 python2.7)

$ python mcve.py 
Traceback (most recent call last):
  File "mcve.py", line 5, in <module>
    print(df.loc[df['dates'] == date])
  File "/opt/miniconda2/lib/python2.7/site-packages/pandas/core/ops.py", line 879, in wrapper
    res = na_op(values, other)
  File "/opt/miniconda2/lib/python2.7/site-packages/pandas/core/ops.py", line 808, in na_op
    y = libindex.convert_scalar(x, _values_from_object(y))
  File "pandas/_libs/index.pyx", line 494, in pandas._libs.index.convert_scalar
  File "pandas/_libs/index.pyx", line 509, in pandas._libs.index.convert_scalar
ValueError: cannot set a Timestamp with a non-timestamp

watercrossing added a commit to watercrossing/pandas that referenced this issue Nov 9, 2017

@watercrossing watercrossing referenced this issue Nov 9, 2017

Merged

Fix #17965 datetime64 comparison #18188

4 of 4 tasks complete

watercrossing added a commit to watercrossing/pandas that referenced this issue Nov 9, 2017

watercrossing added a commit to watercrossing/pandas that referenced this issue Nov 10, 2017

watercrossing added a commit to watercrossing/pandas that referenced this issue Nov 10, 2017

watercrossing added a commit to watercrossing/pandas that referenced this issue Nov 10, 2017

watercrossing added a commit to watercrossing/pandas that referenced this issue Nov 13, 2017

jreback added a commit that referenced this issue Nov 14, 2017

TomAugspurger added a commit to TomAugspurger/pandas that referenced this issue Dec 8, 2017

TomAugspurger added a commit that referenced this issue Dec 11, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment