Skip to content

Avoid index-to-column conversion in some DataFrame ops - #15763

Merged
rapids-bot[bot] merged 11 commits into
NVIDIA:branch-24.06from
mroeschke:ref/misc/materilization
May 21, 2024
Merged

Avoid index-to-column conversion in some DataFrame ops#15763
rapids-bot[bot] merged 11 commits into
NVIDIA:branch-24.06from
mroeschke:ref/misc/materilization

Conversation

@mroeschke

Copy link
Copy Markdown
Contributor

Description

xref #15494

  • For Index.str, check the dtype instead of the underlying column type (which would materialize RangeIndex)
  • For set_index, don't immediately convert passed objects to column until necessary
  • For _make_operands_and_index_for_binop, don't create pandas object more than once

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@mroeschke mroeschke added Python Affects Python cuDF API. improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels May 15, 2024
@mroeschke
mroeschke requested a review from a team as a code owner May 15, 2024 22:13
Comment thread python/cudf/cudf/core/dataframe.py Outdated
Comment thread python/cudf/cudf/core/dataframe.py Outdated
Comment thread python/cudf/cudf/core/dataframe.py Outdated
other.index.to_pandas()
)
)
and not (

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to revert my suggestion, because from the diff I thought everything below was inside this if block, but they are outside of it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just had to move the walrus condition to the top so the if branch doesn't short circuit without defining these variables

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving the condition looked to have done the trick and all green here

@galipremsagar

Copy link
Copy Markdown
Contributor

/merge

@rapids-bot
rapids-bot Bot merged commit d78d565 into NVIDIA:branch-24.06 May 21, 2024
@mroeschke
mroeschke deleted the ref/misc/materilization branch May 21, 2024 17:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Improvement / enhancement to an existing function non-breaking Non-breaking change Python Affects Python cuDF API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants