Skip to content

Conversation

jbrockmendel
Copy link
Member

cc @jorisvandenbossche we briefly discussed at the sprint the idea that dtype.concat is a weird place to define these functions.

This PR takes the subset of dtypes.concat methods that are a) private and b) equivalent to klass._concat_same_dtype for some klass and moves the implementation to the appropriate class.

The categorical one is left in place for now because a) it is public and b) it'd be a pretty big move in and of itself.

"""
# must be overridden in specific classes
return _concat._concat_index_asobject(to_concat, name)
klasses = (ABCDatetimeIndex, ABCTimedeltaIndex, ABCPeriodIndex, ExtensionArray)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should there be references to specific Index types in pandas/core/indexes/base.py?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've already got a few; in this case Index is serving as less of a base class and more of a ObjectIndex.

Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR takes the subset of dtypes.concat methods that are a) private and b) equivalent to klass._concat_same_dtype for some klass and moves the implementation to the appropriate class.

dtypes.concat was meant to hold all the concat things so they are in 1 place. Now we have some in Index and some there. Is this pr meant as a precusor to moar consolidation? The reason we moved these in the first place was that there were multiple definitions instead of a single canonical one.

Since these are used exactly once I guess its ok to move them back, but this kind of moves us backwards here IMHO.

I do like the idea of consolidation, but would like to here what where you think this can go first.

@jreback jreback added the Reshaping Concat, Merge/Join, Stack/Unstack, Explode label Aug 4, 2019
@jbrockmendel
Copy link
Member Author

dtypes.concat was meant to hold all the concat things so they are in 1 place.

Totally on board with the principle here. But dtypes is a weird place for what is a collection of array functions. And in the cases that are being moved here, they don't really share any code, they're just class methods that are being implemented away from the relevant classes.

Is this pr meant as a precusor to moar consolidation?

Yes. union_categorical and concat_categorical are left in place because just-do-the-incorrectly-private-stuff seemed like a good stopping point. But these will belong in a location that reflects the fact that they are array functions, possibly with some of the other helper functions in arrays.categorical that are used in not-just-Categorical places.

@jreback jreback added this to the 1.0 milestone Aug 5, 2019
@jreback jreback merged commit 4c76505 into pandas-dev:master Aug 5, 2019
@jreback
Copy link
Contributor

jreback commented Aug 5, 2019

thanks for the expl @jbrockmendel

I think a good goal then would be to remove completely pandas.core.dtypes.concat and move to appropriate places the remaining.

@jbrockmendel jbrockmendel deleted the concat branch August 5, 2019 14:30
quintusdias pushed a commit to quintusdias/pandas_dev that referenced this pull request Aug 16, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Reshaping Concat, Merge/Join, Stack/Unstack, Explode

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants