Skip to content

Conversation

phofl
Copy link
Member

@phofl phofl commented Sep 17, 2022

  • closes #xxxx (Replace xxxx with the Github issue number)
  • Tests added and passed if fixing a bug or adding a new feature
  • All code checks passed.
  • Added type annotations to new arguments/methods/functions.
  • Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

Nice performance boost

       before           after         ratio
     [b5632fb3]       [ed481f4f]
     <intersection~4>       <intersection>
-      8.81±0.2ms      7.49±0.08ms     0.85  index_object.SetOperations.time_operation('non_monotonic', 'strings', 'symmetric_difference')
-      16.5±0.2ms      12.2±0.05ms     0.74  index_object.SetOperations.time_operation('monotonic', 'date_string', 'intersection')
-      49.7±0.7ms       20.7±0.3ms     0.42  multiindex_object.SetOperations.time_operation('monotonic', 'ea_int', 'intersection')
-      49.9±0.3ms       19.9±0.3ms     0.40  multiindex_object.SetOperations.time_operation('monotonic', 'int', 'intersection')
-         119±1ms       33.2±0.3ms     0.28  multiindex_object.SetOperations.time_operation('monotonic', 'datetime', 'intersection')
-      31.7±0.1ms      4.98±0.05ms     0.16  multiindex_object.SetOperations.time_operation('non_monotonic', 'string', 'intersection')
-      31.8±0.4ms       4.83±0.2ms     0.15  multiindex_object.SetOperations.time_operation('monotonic', 'string', 'intersection')
-      35.5±0.2ms      4.62±0.01ms     0.13  multiindex_object.SetOperations.time_operation('non_monotonic', 'ea_int', 'intersection')
-      34.6±0.4ms       3.35±0.1ms     0.10  multiindex_object.SetOperations.time_operation('non_monotonic', 'int', 'intersection')
-      73.4±0.8ms       3.42±0.1ms     0.05  multiindex_object.SetOperations.time_operation('non_monotonic', 'datetime', 'intersection')

@phofl phofl added MultiIndex NA - MaskedArrays Related to pd.NA and nullable extension arrays labels Sep 17, 2022
@mroeschke mroeschke added this to the 1.6 milestone Sep 19, 2022
@mroeschke mroeschke merged commit 438b957 into pandas-dev:main Sep 19, 2022
@mroeschke
Copy link
Member

Thanks @phofl

@phofl phofl deleted the intersection branch September 20, 2022 08:23
@mroeschke mroeschke modified the milestones: 1.6, 2.0 Oct 13, 2022
noatamir pushed a commit to noatamir/pandas that referenced this pull request Nov 9, 2022
* ENH: MultiIndex.intersection now keeping EA dtypes

* Improve performance

* Improve performance

* Improve performance

* Add test and whatsnew

* Add gh ref

* Fix typing

* Fix typing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
MultiIndex NA - MaskedArrays Related to pd.NA and nullable extension arrays
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants