Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor DataFrame.combine for datetimelike #23079

Closed
TomAugspurger opened this issue Oct 10, 2018 · 0 comments
Closed

Refactor DataFrame.combine for datetimelike #23079

TomAugspurger opened this issue Oct 10, 2018 · 0 comments
Labels
Datetime Datetime data dtype Dtype Conversions Unexpected or buggy dtype conversions
Milestone

Comments

@TomAugspurger
Copy link
Contributor

In DataFrame.combine, we handle datetimelike data special.

needs_i8_conversion_i = needs_i8_conversion(new_dtype)

We attempt to pass a third argument to the combining func. This breaks things like

In [16]: def combiner(x, y): return x

In [17]: a = pd.DataFrame({"A": range(4)})

In [18]: b = pd.DataFrame({"A": pd.date_range('2017', periods=4)})

In [19]: a.combine(a, combiner)
Out[19]:
   A
0  0
1  1
2  2
3  3

In [20]: b.combine(b, combiner)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-20-3271f7709d72> in <module>
----> 1 b.combine(b, combiner)

~/sandbox/pandas/pandas/core/frame.py in combine(self, other, func, fill_value, overwrite)
   5116             needs_i8_conversion_i = needs_i8_conversion(new_dtype)
   5117             if needs_i8_conversion_i:
-> 5118                 arr = func(series, otherSeries, True)
   5119             else:
   5120                 arr = func(series, otherSeries)

TypeError: combiner() takes 2 positional arguments but 3 were given

We could document that, but it still feels hacky. I'd rather that the combiner func infer whether an i8 conversion is needed.

xref #3595

@TomAugspurger TomAugspurger added Datetime Datetime data dtype Dtype Conversions Unexpected or buggy dtype conversions labels Oct 10, 2018
@TomAugspurger TomAugspurger added this to Orthogonal Blockers in DatetimeArray Refactor Oct 18, 2018
TomAugspurger added a commit to TomAugspurger/pandas that referenced this issue Oct 24, 2018
TomAugspurger added a commit to TomAugspurger/pandas that referenced this issue Oct 24, 2018
TomAugspurger added a commit to TomAugspurger/pandas that referenced this issue Oct 24, 2018
@jreback jreback modified the milestones: 0.25.0, 0.24.0 Oct 25, 2018
@TomAugspurger TomAugspurger moved this from Orthogonal Blockers to Done in DatetimeArray Refactor Nov 5, 2018
Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this issue Feb 28, 2019
Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this issue Feb 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Datetime Datetime data dtype Dtype Conversions Unexpected or buggy dtype conversions
Projects
No open projects
Development

No branches or pull requests

2 participants