New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow reading of certain IDL 8.0 .sav files #5801

Merged
merged 2 commits into from Feb 13, 2016

Conversation

Projects
None yet
4 participants
@drhirsch
Copy link
Contributor

drhirsch commented Feb 2, 2016

Some IDL anonymous structures throw an exception needlessly in readsav, the structures read fine when the Exception is turned into a warning. Registration case to verify this patch follows

"""
scipy.io.readsav registration test
illustrates IDL 8.0 .sav file bug workaround
Michael Hirsch
https://scivision.co

This test fails with SciPy 0.16.1 and the latest Git, but will pass
after patching either version with the one line patch below.

In git rev 9e86073, line 283 of scipy.io.readsav, change to
warnings.warn("Error occurred while reading byte array")

"""
from six.moves import urllib
from scipy.io import readsav
from numpy import isclose

def reg_readsav(fn):
    dat = readsav(fn)
    assert isclose(dat['skymap']['full_azimuth'][0][128,128],235.79643)

if __name__ =='__main__':
    url='http://data.phys.ucalgary.ca/sort_by_project/THEMIS/asi/skymaps/new_style/mcgr_20110305/themis_skymap_mcgr_20110305-+_vXX.sav'
    fn = url.rsplit('/',1)[-1]
    try:
        reg_readsav(fn)
    except IOError:
        urllib.request.urlretrieve(url,fn)
        reg_readsav(fn)
Michael Hirsch
allow reading of certain IDL 8.0 .sav files
registration case follows

#/usr/bin/env python3
"""
scipy.io.readsav registration test
illustrates IDL 8.0 .sav file bug workaround
Michael Hirsch
https://scivision.co

This test fails with SciPy 0.16.1 and the latest Git, but will pass
after patching either version with the one line patch below.

In git rev 9e86073, line 283 of scipy.io.readsav, change to
warnings.warn("Error occurred while reading byte array")

"""
from six.moves import urllib
from scipy.io import readsav
from numpy import isclose

def reg_readsav(fn):
    dat = readsav(fn)
    assert isclose(dat['skymap']['full_azimuth'][0][128,128],235.79643)

if __name__ =='__main__':
    url='http://data.phys.ucalgary.ca/sort_by_project/THEMIS/asi/skymaps/new_style/mcgr_20110305/themis_skymap_mcgr_20110305-+_vXX.sav'
    fn = url.rsplit('/',1)[-1]
    try:
        reg_readsav(fn)
    except IOError:
        urllib.request.urlretrieve(url,fn)
        reg_readsav(fn)
@codecov-io

This comment has been minimized.

Copy link

codecov-io commented Feb 2, 2016

@@            master   #5801   diff @@
======================================
  Files          235     235       
  Stmts        43348   43347     -1
  Branches      8165    8165       
  Methods          0       0       
======================================
+ Hit          33723   33725     +2
  Partial       2594    2594       
+ Missed        7031    7028     -3

Review entire Coverage Diff as of e1e60e8

Powered by Codecov. Updated on successful CI builds.

@rgommers

This comment has been minimized.

Copy link
Member

rgommers commented Feb 4, 2016

@scienceopen thanks for the fix. Could you please turn the example above into a self-contained regression test (in scipy/io/tests/test_idl.py)? Then we know that this stays fixed.

michael
@drhirsch

This comment has been minimized.

Copy link
Contributor

drhirsch commented Feb 7, 2016

@rgommers added IDL 8.0 style struct byte array file and regression test that trips error in existing code (but not in my patched version of readsav).

Does that suffice? Thanks for your help.

pv added a commit that referenced this pull request Feb 13, 2016

Merge pull request #5801 from scienceopen/master
ENH: io: allow reading of certain IDL 8.0 .sav files

@pv pv merged commit 9590192 into scipy:master Feb 13, 2016

2 checks passed

codecov/project 77.80% (target 0.01%)
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@pv

This comment has been minimized.

Copy link
Member

pv commented Feb 13, 2016

Thanks, looks good to me. Merged.

@rgommers rgommers added this to the 0.18.0 milestone Feb 13, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment