New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partial Add Reduction Loses Index Combinations #82

Closed
BruceChurch opened this Issue Jan 18, 2018 · 2 comments

Comments

Projects
None yet
2 participants
@BruceChurch

BruceChurch commented Jan 18, 2018

Is this correct behavior for a partial add reduction? Using the canonical example I added a reduction over the first two indices. And the results look strange in that the first index of the resulting 2D sparse matrix are all zeros. Here is a simple example illustrating the point.

Thanks,
Bruce

import numpy as np
import pandas as pd
import sparse 

n = 1000
ndims = 4
nnz = 1000000
coords = np.random.randint(0, n - 1, size=(ndims, nnz))
data = np.random.random(nnz)
x = sparse.COO(coords, data, shape=((n,)*ndims))
z = x.sum(axis=(0,1))
# how many unique combos is the last 2 slots of x? 
df = pd.DataFrame(coords[:2].T)
print df.drop_duplicates().shape
# how many unique combos is the 2 slots of z? 
df2=pd.DataFrame(z.coords.T)
print df2.drop_duplicates().shape
# Shouldn't these match? 
@hameerabbasi

This comment has been minimized.

Collaborator

hameerabbasi commented Jan 18, 2018

@BruceChurch, your test is a little off. This line:

df = pd.DataFrame(coords[:2].T)

Should be

df = pd.DataFrame(coords[2:].T)

Since we have to check the uniques over the axes that we are NOT summing over. That said, PR #83 fixes this issue. Would you mind testing it?

@BruceChurch

This comment has been minimized.

BruceChurch commented Jan 19, 2018

Yes this fixes the problem. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment