New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure densification. #210

Merged
merged 4 commits into from Nov 19, 2018

Conversation

Projects
None yet
4 participants
@hameerabbasi
Copy link
Collaborator

hameerabbasi commented Nov 19, 2018

Add an environment variable to allow configuration of densification.

hameerabbasi added some commits Nov 19, 2018

@codecov

This comment has been minimized.

Copy link

codecov bot commented Nov 19, 2018

Codecov Report

Merging #210 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #210      +/-   ##
==========================================
+ Coverage   97.66%   97.67%   +<.01%     
==========================================
  Files          11       11              
  Lines        1502     1505       +3     
==========================================
+ Hits         1467     1470       +3     
  Misses         35       35
Impacted Files Coverage Δ
sparse/coo/core.py 97.17% <100%> (+0.02%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6e4d35e...80aeb17. Read the comment docs.

hameerabbasi added some commits Nov 19, 2018

@hameerabbasi hameerabbasi merged commit 0b2cc43 into pydata:master Nov 19, 2018

5 checks passed

LGTM analysis: Python No alert changes
Details
ci/circleci Your tests passed on CircleCI!
Details
codecov/patch 100% of diff hit (target 97.66%)
Details
codecov/project 97.67% (+<.01%) compared to 6e4d35e
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
import os

_AUTO_DENSIFICATION_ENABLED = bool(int(os.environ.get('SPARSE_AUTO_DENSIFY', '1')))

This comment has been minimized.

@asmeurer

asmeurer Nov 19, 2018

Member

I would also add

def set_auto_densify(flag):
    """
    Enable or disable automatic densification

    By default it is enabled, unless the environment variable ``SPARSE_AUTO_DENSIFY`` is ``0``.
    
    If automatic densification is disabled, calls like ``np.array(COO)`` will raise ``RuntimeError``.
    """
    global _AUTO_DENSIFICATION_ENABLED
    _AUTO_DENSIFICATION_ENABLED = flag

This comment has been minimized.

@hameerabbasi

hameerabbasi Nov 20, 2018

Collaborator

The problem is that that isn't thread safe. For example, if you do it with Dask or whatever else, it can blow stuff up.

An environment variable loaded at import time is guaranteed to be thread safe.

This comment has been minimized.

@asmeurer

asmeurer Nov 20, 2018

Member

Can we add a warning to the function that it isn't thread safe. I think if it is only called right after import it should be fine too.

This comment has been minimized.

@hameerabbasi

hameerabbasi Nov 20, 2018

Collaborator

I'm worried about exactly the use-case you describe, too. :-) Libraries setting this option and users getting confused about what's happening, unfortunately there's no clear way to make it user facing but not library facing.

This comment has been minimized.

@rgommers

rgommers Nov 21, 2018

unfortunately there's no clear way to make it user facing but not library facing.

That's in the same category as numpy.set_printoptions, np.seterr, private class attributes, etc. - just don't mess with it as a library. It doesn't happen a lot, and if it does the first person who notices files a bug against that library.

@mrocklin

This comment has been minimized.

Copy link
Collaborator

mrocklin commented Nov 20, 2018

@hameerabbasi hameerabbasi deleted the hameerabbasi:configure-densify branch Dec 5, 2018

@asmeurer asmeurer referenced this pull request Dec 12, 2018

Closed

Disallow np.array(COO) #218

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment