Add random() function #41

nils-werner · 2017-12-26T19:35:26Z

I think a random() function is could be pretty handy for generating some quick testing data.

I tried several methods, like generating .coords and .data myself, and creating a COO object from them. In the end it turned out to be the easiest to create a linear scipy.sparse array and simply reshape it to the desired output shape.

mrocklin · 2017-12-26T19:38:25Z

Thanks for this. Some notes:c

This could use a test
Any thoughts on using this to replace our current random_x function currently used in tests? Your solution is likely far more efficient.

nils-werner · 2017-12-26T19:42:50Z

This could use a test

Definitely! What would you like to be tested?

Any thoughts on using this to replace our current random_x function currently used in tests?

Might be worth investigating. I didn't understand what random_x was doing at first glance, so I just ignored it for now.

mrocklin · 2017-12-26T19:48:07Z

Well, we might ensure a variety of things we know about sprase matrices. Here are a few that come to mind:

The shape matches the intended shape
The dtype matches the intended dtype (for a few dtypes)
The density matches the intended density (according to nnz)
Two calls to the same function don't produce the same results
Two calls with the same args and with the same random state seed (we'll need to pass this through) produce equivalent results

mrocklin · 2017-12-26T19:48:35Z

Might be worth investigating. I didn't understand what random_x was doing at first glance, so I just ignored it for now.

It attempts to do the same thing that you're doing here, just much less efficiently, and much less generally :)

hameerabbasi · 2017-12-26T19:53:35Z

The proper way to test a random function is currently a hot debate topic. There seem to be two ways to test it, both which I'm not happy with:

Mock/Monkeypatch the underlying random function to see it is getting the correct arguments. In this case, scipy.sparse.rand.
Run it lots of times until it is statistically certain to be in a certain range, then test for that range.

I will also add that random_x returns a dense Numpy array, and this returns a sparse one.

mrocklin · 2017-12-26T19:57:28Z

I don't think we need to test the random number generator within this. We can rely on SciPy/NumPy to produce a decent distribution of numbers. I'm mostly concerned that we test basic user expectations.

hameerabbasi · 2017-12-27T16:02:08Z

sparse/core.py

+    elements = np.prod(shape)
+
+    return COO.from_scipy_sparse(
+        scipy.sparse.rand(elements, 1, density, dtype=dtype)


Here, it would be useful to specify format='coo' explicitly. Might speed up the conversion.

Maybe I am missing something, but scipy.sparse.rand() defaults to format='coo'...

hameerabbasi · 2017-12-27T16:04:29Z

(we'll need to pass this through)

It looks like the version of scipy.sparse.rand that supports passing through a random state is called scipy.sparse.random and only supports floating point dtypes. So we might lose something if we implement this check.

mrocklin · 2017-12-27T17:04:18Z

It looks like the version of scipy.sparse.rand that supports passing through a random state is called scipy.sparse.random and only supports floating point dtypes. So we might lose something if we implement this check.

That's unfortunate

nils-werner · 2017-12-29T23:13:12Z

It looks like the version of scipy.sparse.rand that supports passing through a random state is called scipy.sparse.random

Really? Looking at the reference it says

scipy.sparse.rand(m, n, density=0.01, format='coo', dtype=None, random_state=None)

and

scipy.sparse.random(m, n, density=0.01, format='coo', dtype=None, random_state=None, data_rvs=None)

Note the random_state kwarg for both functions.

hameerabbasi · 2017-12-29T23:19:06Z

Ah, my mistake, the notes on rand were misleading.

Similar function that allows a user-specified random data source.

nils-werner · 2017-12-31T11:29:34Z

I have replaced random_x and random_x_bool with calls to sparse.random in all the tests. However, I stumbled across a few oddities that need to be discussed. See code review.

hameerabbasi · 2017-12-31T11:51:34Z

sparse/tests/test_core.py

@@ -9,28 +9,6 @@
 import sparse


Please remove random here, it's unneeded.

hameerabbasi · 2017-12-31T11:52:56Z

sparse/tests/test_core.py

-        x[tuple(random.randint(0, d - 1) for d in x.shape)] = True
-    return x
-
-
 @pytest.mark.parametrize('reduction,kwargs', [


Add an extra newline here.

hameerabbasi · 2017-12-31T11:53:53Z

The issue you were having here earlier was already fixed with #56. I've added comments that should help fix the flake8 errors.

hameerabbasi · 2017-12-31T11:54:48Z

I'm good to merge as soon as flake8 is fixed.

Edit: It'd be helpful to add the test @mrocklin described, though.

nils-werner · 2017-12-31T12:37:48Z

Don't merge yet, the tests still randomly fail due to the input data

hameerabbasi · 2017-12-31T12:52:01Z

You might want to rebase this on master and see if the tests still fail. If we're talking about the one I saw fail (test_reductions) then it was fixed in #56.

hameerabbasi · 2017-12-31T13:38:10Z

Since it's just failing for the np.float16 dtype, I'm pretty sure these are just floating point errors. np.allclose checks for a relative error of 1.0e-5 (see here) and float16 is nowhere near that accurate. You might want to pass an additional argument called rtol to assert_eq that defaults to 1.0e-5, increase that in this case. I think this fails because we're moving from straight integers stored in a floating-point dtype to real floating-points... Which increases the likelihood of random errors.

nils-werner · 2017-12-31T14:51:52Z

If I raise atol to 1e-2 (which I find uncomfortably high) I stop seeing random errors...

hameerabbasi · 2017-12-31T15:03:28Z

Great! Fix flake8 and add a test for random and we should be good to merge! I could do that too, if you like!

Thanks for your work on all this.

Edit: You just need to run pip install flake8 (or the conda equivalent) and run flake8 inside the root of the local git repo.

hameerabbasi · 2017-12-31T15:13:50Z

float16 has just 10 bits of floating point precision, out of which one is essentially a sign bit. That leaves 9 bits. 2 ** -9 = 1.953125e-3. Account for the fact that the errors can accumulate, and I think we're good with atol=1.0e-2.

Of course, this is relative tolerance, but since the maximum value of random is 1.0-ish, I'm pretty sure the words carry over.

nils-werner · 2017-12-31T15:37:55Z

While playing around with ways of comparing arrays I realized that COO(scipy.sparse.random(...)) always returns unsorted indices. Do we want them to be never sorted/always sorted/optionally sorted?

hameerabbasi · 2017-12-31T15:41:33Z

It would be nice to have an option sorted which defaults to False. I actually think sorted=False is better because we don't want our methods to assume sorted indices, we want them to work regardless of whether the input is or isn't sorted.

hameerabbasi · 2018-01-01T00:36:55Z

During testing I found that the dtype argument in scipy.random.sparse has no effect. I was considering removing it.

Edit: I also added tests and rebased this onto master.

mrocklin · 2018-01-01T03:43:03Z

Long term we might consider copying scipy's logic to produce the coordinates but then create our own data.

hameerabbasi · 2018-01-01T04:29:04Z

Do you think this is good to merge? It's good from my side. I took a look at SciPy's code, it's got a lot of unnecessary branches and I think we're better off just using it.

nils-werner · 2018-01-01T12:08:21Z

Or we could use the coordinates from SciPy and then do

self.data = numpy.random.rand(elements, dtype=dtype)

hameerabbasi · 2018-01-01T12:37:13Z

numpy.random.rand doesn't seem to have a dtype parameter either.

nils-werner · 2018-01-01T12:55:24Z

That's right... no Idea why I thought it had one...

nils-werner · 2018-01-01T15:10:42Z

Maybe this is starting to overengineer the solution a bit, but we could allow for a data_callback kwarg, and users could inject their own random data generator into the function call:

sparse.random((2, 3, 5), density=0.5)  # float
sparse.random((2, 3, 5), density=0.5, data_callback=lambda x: np.random.choice([True, False], size=x))  # True and False
sparse.random((2, 3, 5), density=0.5, data_callback=lambda x: np.random.randint(10, 20, size=x)).todense()  # Integers from 10 to 20
sparse.random((2, 3, 5), density=0.5, data_callback=lambda x: np.full(x, True))  # all True

hameerabbasi · 2018-01-01T15:26:36Z

No, I actually really like this solution. However, I think it should be in a separate method rather than in sparse.random. I think we should mimic scipy.sparse.random with sparse.random, and add another utility method random_callback (my brain is crapping out on a better name). However, that's for another day... Maybe another pull request?

hameerabbasi · 2018-01-01T16:22:24Z

Merging if there are no comments until 21:00 German time. This can hold tests on other branches back, and they may need to be rebased on this, so I don't want to hold this out too long.

nils-werner · 2018-01-01T16:48:10Z

Actually, scipy.sparse.random() does exactly that, and calls the parameter data_rvs. However what they don't do is change the output array dtype.

I've pushed a commit that implements the parameter, and tests it with a few possible shapes, densities and RVS's (one taken from the scipy.sparse.random() docs)

hameerabbasi · 2018-01-01T16:52:38Z

Hmmm... This seems rather inefficient and wasteful to generate the data twice. Maybe it's worth it to copy this part of the Scipy code if we can guarantee consistency internally.

hameerabbasi · 2018-01-01T17:30:01Z

@nils-werner Any more changes? If not, we should merge this.

nils-werner · 2018-01-01T17:33:30Z

Slow down... we've just doubled the LOC in random() a minute ago, and nobody had the time to review the code yet... :-)

hameerabbasi · 2018-01-01T17:34:44Z

Good point. I just really don't want to put off changing the test code in the other branches.

mrocklin · 2018-01-01T20:37:01Z

sparse/utils.py

@@ -2,7 +2,7 @@
 from .core import COO


-def assert_eq(x, y):
+def assert_eq(x, y, rtol=1.e-5, atol=1.e-8):


Should we just pass through **kwargs here?

Yes, that seems like a better option.

mrocklin · 2018-01-01T20:38:16Z

Overall this seems great to me. It looks like we're able to handle most cases and we're staying within the scipy.sparse API.

nils-werner force-pushed the random branch from 02eb6ae to 0eee01b Compare December 26, 2017 19:40

hameerabbasi reviewed Dec 27, 2017

View reviewed changes

nils-werner force-pushed the random branch from 1badb20 to a8ae22d Compare December 30, 2017 09:15

hameerabbasi reviewed Dec 31, 2017

View reviewed changes

sparse/tests/test_core.py

@@ -9,28 +9,6 @@

import sparse

Copy link

Collaborator

hameerabbasi Dec 31, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove random here, it's unneeded.

hameerabbasi reviewed Dec 31, 2017

View reviewed changes

nils-werner force-pushed the random branch from 40d97a9 to 6222628 Compare December 31, 2017 12:58

nils-werner added 3 commits January 1, 2018 01:33

Add random() function

3946ea3

random(): removed unused nnz variable

e6fe14b

random(): random_state parameter

5b0c14b

nils-werner and others added 3 commits January 1, 2018 01:34

Lower precision for float16 reductions

ad65097

fixed flake8 errors

880504e

Add canonical_order kwarg and add tests.

0471fcc

hameerabbasi force-pushed the random branch from b3c2637 to 0471fcc Compare January 1, 2018 00:35

Remove dtype arguent because it did nothing.

8012ce3

random(): data_rvs parameter

4a757f5

nils-werner and others added 4 commits January 1, 2018 18:10

random(): Example call with data_rvs and random_state

151aa6c

Add remove duplicate data generation.

9a84fbc

Merge remote-tracking branch 'nils-werner/random' into random

c799ba4

Fix Python 2.7 tests.

a885f26

Indices must be integers, not floats

b7b8706

mrocklin reviewed Jan 1, 2018

View reviewed changes

hameerabbasi added 2 commits January 1, 2018 21:58

Passed through all kwargs instead of specific ones.

90f3f89

Less aggressive dtype. May save memory in a lot of cases.

ee8f3b7

hameerabbasi merged commit 462f606 into pydata:master Jan 2, 2018

Add random() function #41

Add random() function #41

Conversation

nils-werner commented Dec 26, 2017

mrocklin commented Dec 26, 2017

nils-werner commented Dec 26, 2017

mrocklin commented Dec 26, 2017

mrocklin commented Dec 26, 2017

hameerabbasi commented Dec 26, 2017

mrocklin commented Dec 26, 2017

hameerabbasi Dec 27, 2017

Choose a reason for hiding this comment

nils-werner Dec 29, 2017 • edited

Choose a reason for hiding this comment

hameerabbasi commented Dec 27, 2017 • edited

mrocklin commented Dec 27, 2017

nils-werner commented Dec 29, 2017 • edited

hameerabbasi commented Dec 29, 2017

nils-werner commented Dec 31, 2017

hameerabbasi Dec 31, 2017

Choose a reason for hiding this comment

hameerabbasi Dec 31, 2017

Choose a reason for hiding this comment

hameerabbasi commented Dec 31, 2017

hameerabbasi commented Dec 31, 2017 • edited

nils-werner commented Dec 31, 2017

hameerabbasi commented Dec 31, 2017

hameerabbasi commented Dec 31, 2017 • edited

nils-werner commented Dec 31, 2017 • edited

hameerabbasi commented Dec 31, 2017 • edited

hameerabbasi commented Dec 31, 2017 • edited

nils-werner commented Dec 31, 2017 • edited

hameerabbasi commented Dec 31, 2017

hameerabbasi commented Jan 1, 2018 • edited

mrocklin commented Jan 1, 2018

hameerabbasi commented Jan 1, 2018

nils-werner commented Jan 1, 2018

hameerabbasi commented Jan 1, 2018

nils-werner commented Jan 1, 2018

nils-werner commented Jan 1, 2018

hameerabbasi commented Jan 1, 2018 • edited

hameerabbasi commented Jan 1, 2018

nils-werner commented Jan 1, 2018 • edited

hameerabbasi commented Jan 1, 2018

hameerabbasi commented Jan 1, 2018

nils-werner commented Jan 1, 2018 • edited

hameerabbasi commented Jan 1, 2018

mrocklin Jan 1, 2018

Choose a reason for hiding this comment

hameerabbasi Jan 1, 2018

Choose a reason for hiding this comment

mrocklin commented Jan 1, 2018

nils-werner Dec 29, 2017 •

edited

hameerabbasi commented Dec 27, 2017 •

edited

nils-werner commented Dec 29, 2017 •

edited

hameerabbasi commented Dec 31, 2017 •

edited

hameerabbasi commented Dec 31, 2017 •

edited

nils-werner commented Dec 31, 2017 •

edited

hameerabbasi commented Dec 31, 2017 •

edited

hameerabbasi commented Dec 31, 2017 •

edited

nils-werner commented Dec 31, 2017 •

edited

hameerabbasi commented Jan 1, 2018 •

edited

hameerabbasi commented Jan 1, 2018 •

edited

nils-werner commented Jan 1, 2018 •

edited

nils-werner commented Jan 1, 2018 •

edited