Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow reading of certain IDL 8.0 .sav files #5801

Merged
merged 2 commits into from Feb 13, 2016
Merged

allow reading of certain IDL 8.0 .sav files #5801

merged 2 commits into from Feb 13, 2016

Conversation

@drhirsch
Copy link
Contributor

@drhirsch drhirsch commented Feb 2, 2016

Some IDL anonymous structures throw an exception needlessly in readsav, the structures read fine when the Exception is turned into a warning. Registration case to verify this patch follows

"""
scipy.io.readsav registration test
illustrates IDL 8.0 .sav file bug workaround
Michael Hirsch
https://scivision.co

This test fails with SciPy 0.16.1 and the latest Git, but will pass
after patching either version with the one line patch below.

In git rev 9e86073, line 283 of scipy.io.readsav, change to
warnings.warn("Error occurred while reading byte array")

"""
from six.moves import urllib
from scipy.io import readsav
from numpy import isclose

def reg_readsav(fn):
    dat = readsav(fn)
    assert isclose(dat['skymap']['full_azimuth'][0][128,128],235.79643)

if __name__ =='__main__':
    url='http://data.phys.ucalgary.ca/sort_by_project/THEMIS/asi/skymaps/new_style/mcgr_20110305/themis_skymap_mcgr_20110305-+_vXX.sav'
    fn = url.rsplit('/',1)[-1]
    try:
        reg_readsav(fn)
    except IOError:
        urllib.request.urlretrieve(url,fn)
        reg_readsav(fn)
registration case follows

#/usr/bin/env python3
"""
scipy.io.readsav registration test
illustrates IDL 8.0 .sav file bug workaround
Michael Hirsch
https://scivision.co

This test fails with SciPy 0.16.1 and the latest Git, but will pass
after patching either version with the one line patch below.

In git rev 9e86073, line 283 of scipy.io.readsav, change to
warnings.warn("Error occurred while reading byte array")

"""
from six.moves import urllib
from scipy.io import readsav
from numpy import isclose

def reg_readsav(fn):
    dat = readsav(fn)
    assert isclose(dat['skymap']['full_azimuth'][0][128,128],235.79643)

if __name__ =='__main__':
    url='http://data.phys.ucalgary.ca/sort_by_project/THEMIS/asi/skymaps/new_style/mcgr_20110305/themis_skymap_mcgr_20110305-+_vXX.sav'
    fn = url.rsplit('/',1)[-1]
    try:
        reg_readsav(fn)
    except IOError:
        urllib.request.urlretrieve(url,fn)
        reg_readsav(fn)
@codecov-io
Copy link

@codecov-io codecov-io commented Feb 2, 2016

@@            master   #5801   diff @@
======================================
  Files          235     235       
  Stmts        43348   43347     -1
  Branches      8165    8165       
  Methods          0       0       
======================================
+ Hit          33723   33725     +2
  Partial       2594    2594       
+ Missed        7031    7028     -3

Review entire Coverage Diff as of e1e60e8

Powered by Codecov. Updated on successful CI builds.

@rgommers
Copy link
Member

@rgommers rgommers commented Feb 4, 2016

@scienceopen thanks for the fix. Could you please turn the example above into a self-contained regression test (in scipy/io/tests/test_idl.py)? Then we know that this stays fixed.

michael
@drhirsch
Copy link
Contributor Author

@drhirsch drhirsch commented Feb 7, 2016

@rgommers added IDL 8.0 style struct byte array file and regression test that trips error in existing code (but not in my patched version of readsav).

Does that suffice? Thanks for your help.

pv added a commit that referenced this pull request Feb 13, 2016
ENH: io: allow reading of certain IDL 8.0 .sav files
@pv pv merged commit 9590192 into scipy:master Feb 13, 2016
2 checks passed
2 checks passed
codecov/project 77.80% (target 0.01%)
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@pv
Copy link
Member

@pv pv commented Feb 13, 2016

Thanks, looks good to me. Merged.

@rgommers rgommers added this to the 0.18.0 milestone Feb 13, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants