API: add Series.dt delegator for datetimelike methods (GH7207) #7953

jreback · 2014-08-07T16:59:07Z

In [1]: s = Series(date_range('20130101',periods=3))

In [2]: s.dt.
s.dt.date              s.dt.dayofweek         s.dt.hour              s.dt.is_month_start    s.dt.is_quarter_start  s.dt.is_year_start     s.dt.minute            s.dt.nanosecond        s.dt.second            s.dt.week              s.dt.year              
s.dt.day               s.dt.dayofyear         s.dt.is_month_end      s.dt.is_quarter_end    s.dt.is_year_end       s.dt.microsecond       s.dt.month             s.dt.quarter           s.dt.time              s.dt.weekofyear        

In [3]: s.dt.year
Out[3]: array([2013, 2013, 2013])

In [4]: s.dt.hour
Out[4]: array([0, 0, 0])

In [5]: Series(np.arange(5)).dt
TypeError: Can only use .dt accessor with datetimelike values

Tab completion is specific to the type of wrapped delegate (DatetimeIndex or PeriodIndex)

In [5]: p = Series(period_range('20130101',periods=3,freq='D').asobject)

In [6]: p.dt.
p.dt.day         p.dt.dayofweek   p.dt.dayofyear   p.dt.hour        p.dt.minute      p.dt.month       p.dt.quarter     p.dt.qyear       p.dt.second      p.dt.week        p.dt.weekofyear  p.dt.year

jankatins · 2014-08-07T22:48:22Z

pandas/core/index.py

+    def __dir__(self):
+        return sorted(list(set(self.ops)))
+
+def _maybe_to_datetimelike_index(data, copy=False):


Is that still a index, now it returns a Series? If not the name should probably be changed

jorisvandenbossche · 2014-08-08T12:28:36Z

Looks good! Thanks for this. The only thing I was thinking to have it named DatetimeAttributes more in line with StringMethods, but doesn't matter.

Only should need some dos (whatsnew, addition to the API (http://pandas.pydata.org/pandas-docs/stable/api.html#time-series-related), and somewhere else in the docs)

jreback · 2014-08-08T12:32:06Z

yeh could change the name

jreback · 2014-08-08T14:24:12Z

@jorisvandenbossche ok, this nicely builds the API (similar to StringMethods).

jankatins · 2014-08-08T15:50:44Z

doc/source/basics.rst

+.dt accessor
+~~~~~~~~~~~~
+
+``Series`` has an accessor to succintly return datetime like properties for the *values* of the Series, if its a datetime/period like Series. (:issue:`7207`)


succinctly? I don't think there should be a GH link in the basics section?

jreback · 2014-08-08T17:21:20Z

@jorisvandenbossche

when accessing a doc-string for a property

s.dt.hour? it 'works' in that the doc-string is printed, but all of the property info is also printed, any way to prevent that? (is my function creator wrong?)

In [1]: s = Series(date_range('20130101',periods=5))

In [2]: s.dt.hour
Out[2]: 
0    0
1    0
2    0
3    0
4    0
dtype: int64

In [3]: s.dt.hour?
Type:            property
String form:     <property object at 0x47998e8>
Docstring:       The hours of the datetime
Class docstring:
property(fget=None, fset=None, fdel=None, doc=None) -> property attribute

fget is a function to be used for getting an attribute value, and likewise
fset is a function for setting, and fdel a function for del'ing, an
attribute.  Typical use is to define a managed attribute x:
class C(object):
    def getx(self): return self._x
    def setx(self, value): self._x = value
    def delx(self): del self._x
    x = property(getx, setx, delx, "I'm the 'x' property.")

Decorators make defining new properties or modifying existing ones easy:
class C(object):
    @property
    def x(self): return self._x
    @x.setter
    def x(self, value): self._x = value
    @x.deleter
    def x(self): del self._x

jreback · 2014-08-08T17:33:35Z

pandas/tseries/datetimelike.py

+    raise TypeError("cannot convert an object of type {0} to a datetimelike index".format(type(data)))
+
+# facilitate the properties on the wrapped ops
+def _add_accessors(cls):


@JanSchulz see here

jreback · 2014-08-08T19:49:11Z

if you have a specific method, then just define it directly (after all this is just a convience to avoid writing each method manuall). which are you referring?

jankatins · 2014-08-08T20:31:23Z

I've added the possibility for a setter in jankatins@1b52a71

jreback · 2014-08-08T20:32:29Z

gr8; I'll incorporate that and repush

jankatins · 2014-08-08T20:35:09Z

Re "different methods": I just noticed that when I wrote my implementation of "CategoricalProperties" (in the end it was not needed).

jreback · 2014-08-08T20:38:18Z

@JanSchulz updated to incorporate the get/set

jorisvandenbossche · 2014-08-08T22:58:46Z

@jreback for the property docs, I think that this is an IPython issue (see ipython/ipython#1555, and fixed by @immerrr in ipython/ipython#4827 which will be in IPython 3.0 I think).
If you do the plain python help way help() it does return only the docstring.

jorisvandenbossche · 2014-08-08T23:03:54Z

doc/source/basics.rst

+.dt accessor
+~~~~~~~~~~~~
+
+``Series`` has an accessor to succinctly return datetime like properties for the *values* of the Series, if its a datetime/period like Series. (:issue:`7207`)


link to issue is not needed here I think (it is in the whatsnew)

jorisvandenbossche · 2014-08-08T23:15:31Z

Nice docs! Added some but very minor comments.

hayd · 2014-08-09T19:55:27Z

pandas/core/base.py

+    def _delegate_property_get(self, name):
+        raise NotImplementedError("PandasDelegate is an abstract base class: _delegate_property_get """)
+
+    def _delegate_property_set(self, name):


potentially this should have new_values arg, so that it raises correctly (atm it's a TypeError, wrong number of args).

Though I don't quite follow how setters will work... would we be allowing things like s.dt.year[0] = 2000 and weird things like this (atm that'll fail silently but perhaps it's possible to make it raise :s).

it's not applicable for indexs (though for series it could work) - it's defined as a base fore Categoical namespace

Hmm, IMO ABC error is going to be a bit confusing when people try and do this (on this branch):

In [11]: s = pd.Series(pd.to_datetime(['2012', '2013'])) In [12]: s.dt.year = [1000, 2000] # if add the new_values this'll be a NotImplemented ABC error. TypeError: _delegate_property_set() takes exactly 2 arguments (3 given) In [13]: dt = pd.to_datetime(['2012', '2013'] ) In [14]: dt.year = [1000, 2000] # this error makes sense AttributeError: can't set attribute

Note: Assigning to the ~~copy~~ temporary dt.year[0] = 2000 always (has) silently failed.

good point
these should always fail

I don't think we should allow modifications here
too complicated / messy and not necessary

(maybe I. the future)

I'll fix the error though

Great! I agree, would be messy, difficult to get right* and slow too.

* maybe even not possible e.g. leap years days and stuff.

In [1]: s = Series(date_range('20130101',periods=3)) In [2]: s.dt.hour = 5 ValueError: cannot set a datetimelike property In [3]: s.dt.hour[0] = 5 /usr/local/bin/ipython:1: SettingWithCopyWarning: A value is trying to be set on an immutable datetimelike object

jankatins · 2014-08-10T15:05:03Z

A , use this-and-that. would be nice :-)

jreback · 2014-08-10T15:31:18Z

ok in the last

what's a 'use this or that ' mean?

jankatins · 2014-08-10T16:51:37Z

Reading this as a "phd student who does some data munging" would leave me confused:
"A value is trying to be set on an immutable datetimelike object"

So something like this:
"Changing values on '%s' is not supported, change the values on the original Series." % name

hayd · 2014-08-10T18:16:37Z

What about:

Modifications to a property of a datetimelike object are not supported and are discarded. Change values on the original {0}.

This is a great PR, looking forward to this!

jankatins · 2014-08-10T18:32:42Z

@hayd Much better!

jreback · 2014-08-10T21:28:35Z

@JanSchulz @hayd updated to your text for the error messages

@jorisvandenbossche @cpcloud

cc @rosnfeld
cc @mtkni

any other comments?

jorisvandenbossche · 2014-08-10T21:33:33Z

doc/source/basics.rst

+.dt accessor
+~~~~~~~~~~~~
+
+``Series`` has an accessor to succinctly return datetime like properties for the *values* of the Series, if its a datetime/period like Series.


its -> it's

jorisvandenbossche · 2014-08-10T21:38:08Z

Looking good to me (just two typo/grammar things), and this is a nice enhancement!

rosnfeld · 2014-08-10T21:50:07Z

I pulled the changes and played around with it, looks very good to me. The only minor improvement I'd suggest is a tiny bit more documentation on the actual DateTimeProperties/PeriodProperties code, i.e. compare:

In [5]: s.str?
Type:       StringMethods
String Form:<pandas.core.strings.StringMethods object at 0x3c23410>
File:       /home/andrew/git/pandas-rosnfeld/pandas/core/strings.py
Docstring:
Vectorized string functions for Series. NAs stay NA unless handled
otherwise by a particular method. Patterned after Python's string methods,
with some inspiration from R's stringr package.

Examples
--------
>>> s.str.split('_')
>>> s.str.replace('_', '')

In [6]: s.dt?
Type:       DatetimeProperties
String Form:<pandas.tseries.common.DatetimeProperties object at 0x3c06150>
File:       /home/andrew/git/pandas-rosnfeld/pandas/tseries/common.py
Docstring:  Manages a DatetimeIndex Delegate

jreback · 2014-08-10T21:50:53Z

@rosnfeld makes sense, I will emulate

jreback · 2014-08-10T21:59:58Z

In [3]: s.dt?
Type:        DatetimeProperties
String form: <pandas.tseries.common.DatetimeProperties object at 0x3a76650>
File:        /mnt/home/jreback/pandas/pandas/tseries/common.py
Docstring:
Accessor object for datetimelike properties of the Series values.

Examples
--------
>>> s.dt.hour
>>> s.dt.second
>>> s.dt.quarter

Returns a Series indexed like the original Series.
Raises TypeError if the Series does not contain datetimelike values.

In [4]: quit()

CLN: make _add_delegate_accessors generic

rosnfeld · 2014-08-10T22:05:04Z

Nice - thanks!

API: add Series.dt delegator for datetimelike methods (GH7207)

jreback · 2014-08-10T22:25:41Z

bombs away.

jreback added API Design labels Aug 7, 2014

jreback mentioned this pull request Aug 7, 2014

API: revisit adding datetime-like ops in Series #7207

Closed

jreback added this to the 0.15.0 milestone Aug 7, 2014

jankatins reviewed Aug 7, 2014
View reviewed changes

jankatins reviewed Aug 8, 2014
View reviewed changes

jreback reviewed Aug 8, 2014
View reviewed changes

jorisvandenbossche reviewed Aug 8, 2014
View reviewed changes

hayd reviewed Aug 9, 2014
View reviewed changes

jorisvandenbossche reviewed Aug 10, 2014
View reviewed changes

API: add Series.dt delegator for datetimelike methods (GH7207)

aa1c120

CLN: make _add_delegate_accessors generic

jreback added a commit that referenced this pull request Aug 10, 2014

Merge pull request #7953 from jreback/dt

db8533f

API: add Series.dt delegator for datetimelike methods (GH7207)

jreback merged commit db8533f into pandas-dev:master Aug 10, 2014

API: add Series.dt delegator for datetimelike methods (GH7207) #7953

API: add Series.dt delegator for datetimelike methods (GH7207) #7953

Conversation

jreback commented Aug 7, 2014

Choose a reason for hiding this comment

jorisvandenbossche commented Aug 8, 2014

jreback commented Aug 8, 2014

jreback commented Aug 8, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Aug 8, 2014

Choose a reason for hiding this comment

jreback commented Aug 8, 2014

jankatins commented Aug 8, 2014

jreback commented Aug 8, 2014

jankatins commented Aug 8, 2014

jreback commented Aug 8, 2014

jorisvandenbossche commented Aug 8, 2014

Choose a reason for hiding this comment

jorisvandenbossche commented Aug 8, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jankatins commented Aug 10, 2014

jreback commented Aug 10, 2014

jankatins commented Aug 10, 2014

hayd commented Aug 10, 2014

jankatins commented Aug 10, 2014

jreback commented Aug 10, 2014

Choose a reason for hiding this comment

jorisvandenbossche commented Aug 10, 2014

rosnfeld commented Aug 10, 2014

jreback commented Aug 10, 2014

jreback commented Aug 10, 2014

rosnfeld commented Aug 10, 2014

jreback commented Aug 10, 2014