Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FIX-#6948: Fix groupby when Modin dataframe has several column partitions #6951

Merged
merged 5 commits into from Feb 22, 2024
Merged

FIX-#6948: Fix groupby when Modin dataframe has several column partitions #6951

merged 5 commits into from Feb 22, 2024

Conversation

anmyachev
Copy link
Collaborator

@anmyachev anmyachev commented Feb 20, 2024 •
edited

What do these changes do?

  • first commit message and PR title follow format outlined here

    NOTE: If you edit the PR title to match this format, you need to add another commit (even if it's empty) or amend your last commit for the CI job that checks the PR title to pick up the new PR title.

  • passes flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
  • passes black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
  • signed commit with git commit -s
  • Resolves Poor Performance on TPC-H Queries #6948
  • tests added and passing
  • module layout described at docs/development/architecture.rst is up-to-date

@anmyachev
FIX-modin-project#6948: Fix groupby when Modin dataframe has several …
51d083b 
…column partitions

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@anmyachev
fixes
1fa941e 
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@anmyachev
add test case
814aebb 
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
anmyachev
anmyachev commented Feb 21, 2024
View reviewed changes
modin/pandas/groupby.py Outdated Show resolved Hide resolved
@anmyachev anmyachev marked this pull request as ready for review February 21, 2024 16:45
YarShev
YarShev reviewed Feb 22, 2024
View reviewed changes
modin/pandas/groupby.py Outdated
@@ -904,19 +905,35 @@ def aggregate(self, func=None, *args, engine=None, engine_kwargs=None, **kwargs)

do_relabel = None
if isinstance(func, dict) or func is None:
relabeling_required, func_dict, new_columns, order = reconstruct_func(
# the order from `reconstruct_func` cannot be used correctly if there
# is more than one columnar partition
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you proceed the comment and answer why?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

modin/pandas/test/test_groupby.py Outdated Show resolved Hide resolved
modin/pandas/groupby.py Outdated Show resolved Hide resolved
@anmyachev
address review comments
06a0ee2 
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@YarShev YarShev merged commit 6dfe13f into modin-project:master Feb 22, 2024
37 checks passed
@anmyachev anmyachev deleted the issue6948 branch February 22, 2024 14:25
tochigiv pushed a commit to tochigiv/modin that referenced this pull request Feb 22, 2024
@anmyachev @tochigiv
FIX-modin-project#6948: Fix groupby when Modin dataframe has several …
d1b8444 
…column partitions (modin-project#6951)

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
tochigiv pushed a commit to tochigiv/modin that referenced this pull request Feb 22, 2024
@anmyachev @tochigiv
FIX-modin-project#6948: Fix groupby when Modin dataframe has several …
9eaedb9 
…column partitions (modin-project#6951)

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@YarShev YarShev YarShev approved these changes

@devin-petersohn devin-petersohn Awaiting requested review from devin-petersohn devin-petersohn is a code owner

@mvashishtha mvashishtha Awaiting requested review from mvashishtha mvashishtha is a code owner

@RehanSD RehanSD Awaiting requested review from RehanSD RehanSD is a code owner

@vnlitvinov vnlitvinov Awaiting requested review from vnlitvinov vnlitvinov is a code owner

@dchigarev dchigarev Awaiting requested review from dchigarev dchigarev is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Poor Performance on TPC-H Queries
2 participants
@anmyachev @YarShev