API: datetimeindex subtraction

[jreback]

from SO and previous discussions

currently you cannot subtract 2 datetimeindexes ('-' is actually a set op). This is the ONLY valid numerical type of operation (e.g. you cannot add dti).

We current support pretty much the fully gamut of obvious ops, with dti-dti being the exception.

full list of tests are here: https://github.com/pydata/pandas/blob/master/pandas/tseries/tests/test_base.py#L434

I will propose

dti.sub(dti) to rectify this

thoughts

[jorisvandenbossche]

We have deprecated the set ops on the index using - and + (#8227, #8226), so in the future, when this deprecation has been put in practice, we can use - for subtraction.
But for now, as long using - is not possible, adding dti.sub(dti) seems like a good idea to me.

Are there actually other reasons one would want to use a sub method instead of -? (because that would be a reason to also add add/multiply/... as for Series)

Then on the naming, is there a reason you have for Sereis both sub and subtract? (and also both mult/multiply, div/divide, ..)

[jreback]

@jorisvandenbossche the deprecations don't apply to either Numeric, Datetime, Period, or Timedelta index (IOW, ONLY to Index itself).

So dti-dti is a set op. This is NOT deprecated. It could be changed (and maybe should be?)

[jreback]

hmm, maybe we should deprecate and do the subtraction.

In [1]: dti = pd.date_range('20130101',periods=3)

In [2]: dti-dti
Out[2]: Index([], dtype='object')

In [3]: dti.difference(dti)
Out[3]: Index([], dtype='object')

[jorisvandenbossche]

@jreback Ah, yes, if we agree on the deprecation of -/+ as index ops, then that should be for all Index subclasses, not only for Index itself, otherwise it is not really usefull

[jreback]

ok then i will change DatetimeIndex in 0.16 then to do direct subtraction - makes more sense
(actually pretty easy to 'warn' on the older operation as the index have to be the same length)

[jorisvandenbossche]

@jreback I think that is too early for such an api change IMO. But see also #9095 I just opened

[shoyer]

I really thought we had deprecated -/+ as index ops on all Index types, though I guess we are not currently issuing a warning for these operations (the docs are more ambiguous).

[jreback]

as I commented on the other issue. This is an anomaly and should be fixed (for PeriodIndex) as well. I don't think waiting ANOTHER big release cycle is warranted.

[jorisvandenbossche]

This is with the latest release 0.15.2:

In [1]: dti = pd.date_range('2012-01-01', periods=2)

In [2]: dti - dti
Out[2]: Index([], dtype='object')

In [3]: pd.__version__
Out[3]: '0.15.2'

This is doing a set operation, without a deprecation warning.
So IMO we should not change the behaviour straight away, but first raise a deprecation warning.

[jreback]

the warning prob should have been there in 0.15

so we'll call this an api change then

[jorisvandenbossche]

@jreback I think we should revisit this for 0.16 (or at least the deprecation)

We do want to be able to do numeric operations, so we should deprecate the set operation (in theory this was already deprecated in 0.15, but for datimeindexes, there was not deprecation warning raised)

on master:

In [6]: dti = pd.date_range('2012-01-01', periods=2)

In [7]: dti - dti
Out[7]: Index([], dtype='object')

[jreback]

yes could deprecate the actual subtraction op (which is the only thing left) on dti - dti for 0.16
(as .difference is the way to go)

[jreback]

@jorisvandenbossche this is in 0.19.0rc1 right? (need to check the issue)

[jorisvandenbossche]

Yes, indeed, but this issue was in the end for raising the deprecation warning, so I didn't reference it from the PR (but is could be referenced). PR that changed the setop to timedelta is #14164