Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Series[offsets] + Series[datetime64] fails #19211

Closed
jbrockmendel opened this issue Jan 12, 2018 · 6 comments
Closed

Series[offsets] + Series[datetime64] fails #19211

jbrockmendel opened this issue Jan 12, 2018 · 6 comments
Labels
good first issue Needs Tests Unit test(s) needed to prevent regressions Numeric Operations Arithmetic, Comparison, and Logical operations

Comments

@jbrockmendel
Copy link
Member

ser = pd.Series([pd.Timestamp('2000-1-1'), pd.Timestamp('2000-2-1')])
other = pd.Series([pd.offsets.DateOffset(years=1), pd.offsets.MonthEnd()])

>>> ser + other
0   2001-01-01
1   2000-02-29
dtype: datetime64[ns]

>>> other + ser
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pandas/core/ops.py", line 698, in wrapper
    result = wrap_results(safe_na_op(lvalues, rvalues))
  File "pandas/core/ops.py", line 665, in safe_na_op
    lambda x: op(x, rvalues))
  File "pandas/_libs/algos_common_helper.pxi", line 1212, in pandas._libs.algos.arrmap_object
    result[i] = func(index[i])
  File "pandas/core/ops.py", line 665, in <lambda>
    lambda x: op(x, rvalues))
  File "pandas/tseries/offsets.py", line 352, in __add__
    return self.apply(other)
  File "pandas/tseries/offsets.py", line 77, in wrapper
    result = func(self, other)
  File "pandas/tseries/offsets.py", line 212, in apply
    other = other + self._offset
TypeError: ufunc add cannot use operands with types dtype('<M8[ns]') and dtype('O')

pd.Index(other) + ser is OK, as is other + pd.DatetimeIndex(ser)

@chris-b1
Copy link
Contributor

More fundamental than the arithmetic is that Series/arrays of offsets aren't meaningfully defined as a type, compared to, e.g. Timedelta.

It wouldn't be an unreasonable addition, something that would have occasionally been useful to me, but a pretty large change. Probably less painful after #19174, et al.

@chris-b1
Copy link
Contributor

As you noted, this does actually sort of work (surprise to me!). I suppose may not need to be that invasive, could just involve the dispatch for + on object Series.

In [220]: pd.Index(other) + ser
C:\Users\chris.bartak\AppData\Local\Continuum\Anaconda3\envs\py36\lib\site-packages\pandas\core\ops.py:556: PerformanceWarning: Adding/subtracting array of DateOffsets to Series not vectorized
  "Series not vectorized", PerformanceWarning)
Out[220]: DatetimeIndex(['2001-01-01', '2000-02-29'], dtype='datetime64[ns]', freq=None)

@jbrockmendel
Copy link
Member Author

The direction we’re going with series arithmetic is to dispatch to the relevant Index subclasses (#19024, #18824) implementations, since they handle type checking (and timezones) much more carefully than the current Series implementations.

@jbrockmendel jbrockmendel added the Numeric Operations Arithmetic, Comparison, and Logical operations label Oct 21, 2019
@mroeschke
Copy link
Member

Looks like this works now on master Could use a test

In [32]: ser = pd.Series([pd.Timestamp('2000-1-1'), pd.Timestamp('2000-2-1')])
    ...: other = pd.Series([pd.offsets.DateOffset(years=1), pd.offsets.MonthEnd()])

In [34]: other + ser
/Users/matthewroeschke/pandas-mroeschke/pandas/core/arrays/datetimelike.py:1305: PerformanceWarning: Adding/subtracting array of DateOffsets to DatetimeArray not vectorized
  PerformanceWarning,
Out[34]:
0   2001-01-01
1   2000-02-29
dtype: datetime64[ns]

In [37]: pd.__version__
Out[37]: '1.1.0.dev0+1068.g49bc8d8c9'

@mroeschke mroeschke added good first issue Needs Tests Unit test(s) needed to prevent regressions and removed Bug Numeric Operations Arithmetic, Comparison, and Logical operations Timeseries labels Apr 1, 2020
@ghost
Copy link

ghost commented Jun 4, 2020

take

@github-actions github-actions bot assigned ghost Jun 4, 2020
@ghost ghost mentioned this issue Jun 4, 2020
5 tasks
@jreback jreback added this to the 1.1 milestone Jun 14, 2020
@jreback jreback modified the milestones: 1.1, Contributions Welcome Jul 10, 2020
@jbrockmendel jbrockmendel added the Numeric Operations Arithmetic, Comparison, and Logical operations label Sep 21, 2020
@jbrockmendel
Copy link
Member Author

This is tested by tests.arithmetic.test_datetime64.test_dt64arr_add_sub_offset_array, among others. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Needs Tests Unit test(s) needed to prevent regressions Numeric Operations Arithmetic, Comparison, and Logical operations
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants