Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix internal indices calculation for non-compute partitions #691

Merged
merged 2 commits into from Jun 18, 2019
Merged

Fix internal indices calculation for non-compute partitions #691

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
05c5dd1
Fix internal indices calculation for non-compute partitions
devin-petersohn Jun 18, 2019
f37425e
Address comment
devin-petersohn Jun 18, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
15 changes: 11 additions & 4 deletions modin/engines/base/frame/partition_manager.py
View file
Open in desktop
Expand Up @@ -685,10 +685,17 @@ def internal(block_idx, global_index):
count_for_each_partition = np.array(
[(partition_ids == i).sum() for i in range(len(cumulative))]
).cumsum()
# compute the internal indices and pair those with the partition index.
partition_ids_with_indices = [
(0, internal(0, indices[slice(count_for_each_partition[0])]))
] + [
# Compute the internal indices and pair those with the partition index.
# If the first partition has any values we need to return, compute those
# first to make the list comprehension easier. Otherwise, just append the
# rest of the values to an empty list.
if count_for_each_partition[0] > 0:
first_partition_indices = [
(0, internal(0, indices[slice(count_for_each_partition[0])]))
]
else:
first_partition_indices = []
partition_ids_with_indices = first_partition_indices + [
(
i,
internal(
Expand Down