Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/sample #87

Merged
merged 10 commits into from
Aug 7, 2019
Merged

Feature/sample #87

merged 10 commits into from
Aug 7, 2019

Conversation

michcio1234
Copy link

No description provided.

And change sf_arange fixture to have values from 1 to 10 rather than
from 0 to 9, because 0 is treated as missing in sparse frames.
When the same dsf is computed twice, results may be different
If an argument is a generator, we will call next() on it before passing
it to a function for each partition.
Previously, if sf was all-zero but not empty (shape was > 0),
empty sf was returned anyway.
Starting from 1.16, it is possible to do `np.random.choice([], size=0)`.

We require numpy above 1.16.0, because 1.16.0 had memory leaks.
Tested with 1.16.4.
@kayibal
Copy link

kayibal commented Aug 7, 2019

@michcio1234 Should we keep it this way? With the generator like map_partitions? If you're ok with keeping it feel free to merge!

@michcio1234
Copy link
Author

Yeah, I know it's a kind of magic, but it may turn out to be useful. I don't think it's dangerous. Let's leave it.

@codecov
Copy link

codecov bot commented Aug 7, 2019

Codecov Report

Merging #87 into master will increase coverage by 0.25%.
The diff coverage is 95.45%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master     #87      +/-   ##
=========================================
+ Coverage   89.64%   89.9%   +0.25%     
=========================================
  Files           7       7              
  Lines        1227    1268      +41     
=========================================
+ Hits         1100    1140      +40     
- Misses        127     128       +1
Impacted Files Coverage Δ
sparsity/sparse_frame.py 91.76% <100%> (+0.17%) ⬆️
sparsity/dask/core.py 85.74% <93.93%> (+0.53%) ⬆️
sparsity/dask/io_.py 93.4% <0%> (+1.09%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7ea5f1a...1e8f673. Read the comment docs.

# Conflicts:
#	sparsity/test/test_sparse_frame.py
@michcio1234 michcio1234 merged commit 2f3b19c into master Aug 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants