Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure densification. #210

Merged
merged 4 commits into from Nov 19, 2018
Merged

Conversation

@hameerabbasi
Copy link
Collaborator

@hameerabbasi hameerabbasi commented Nov 19, 2018

Add an environment variable to allow configuration of densification.

@codecov
Copy link

@codecov codecov bot commented Nov 19, 2018

Codecov Report

Merging #210 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #210      +/-   ##
==========================================
+ Coverage   97.66%   97.67%   +<.01%     
==========================================
  Files          11       11              
  Lines        1502     1505       +3     
==========================================
+ Hits         1467     1470       +3     
  Misses         35       35
Impacted Files Coverage Δ
sparse/coo/core.py 97.17% <100%> (+0.02%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6e4d35e...80aeb17. Read the comment docs.

@hameerabbasi hameerabbasi merged commit 0b2cc43 into pydata:master Nov 19, 2018
5 checks passed
5 checks passed
LGTM analysis: Python No alert changes
Details
ci/circleci Your tests passed on CircleCI!
Details
@codecov
codecov/patch 100% of diff hit (target 97.66%)
Details
@codecov
codecov/project 97.67% (+<.01%) compared to 6e4d35e
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
import os

_AUTO_DENSIFICATION_ENABLED = bool(int(os.environ.get('SPARSE_AUTO_DENSIFY', '1')))

This comment has been minimized.

@asmeurer

asmeurer Nov 19, 2018
Member

I would also add

def set_auto_densify(flag):
    """
    Enable or disable automatic densification

    By default it is enabled, unless the environment variable ``SPARSE_AUTO_DENSIFY`` is ``0``.
    
    If automatic densification is disabled, calls like ``np.array(COO)`` will raise ``RuntimeError``.
    """
    global _AUTO_DENSIFICATION_ENABLED
    _AUTO_DENSIFICATION_ENABLED = flag

This comment has been minimized.

@hameerabbasi

hameerabbasi Nov 20, 2018
Author Collaborator

The problem is that that isn't thread safe. For example, if you do it with Dask or whatever else, it can blow stuff up.

An environment variable loaded at import time is guaranteed to be thread safe.

This comment has been minimized.

@asmeurer

asmeurer Nov 20, 2018
Member

Can we add a warning to the function that it isn't thread safe. I think if it is only called right after import it should be fine too.

This comment has been minimized.

@hameerabbasi

hameerabbasi Nov 20, 2018
Author Collaborator

I'm worried about exactly the use-case you describe, too. :-) Libraries setting this option and users getting confused about what's happening, unfortunately there's no clear way to make it user facing but not library facing.

This comment has been minimized.

@rgommers

rgommers Nov 21, 2018

unfortunately there's no clear way to make it user facing but not library facing.

That's in the same category as numpy.set_printoptions, np.seterr, private class attributes, etc. - just don't mess with it as a library. It doesn't happen a lot, and if it does the first person who notices files a bug against that library.

@mrocklin
Copy link
Collaborator

@mrocklin mrocklin commented Nov 20, 2018

@hameerabbasi hameerabbasi deleted the hameerabbasi:configure-densify branch Dec 5, 2018
@asmeurer asmeurer mentioned this pull request Dec 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants