Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Seek and read from closed MemoryIO #1539

Closed
braingram opened this issue May 4, 2023 · 2 comments
Closed

Seek and read from closed MemoryIO #1539

braingram opened this issue May 4, 2023 · 2 comments

Comments

@braingram
Copy link
Contributor

ASDF will read from a 'closed' generic_io.MemoryIO (like what is created when reading from an io.BytesIO instance).

Running the following:

import io
import asdf
import numpy

b = io.BytesIO()
arrs = [numpy.arange(10) + i for i in range(10)]
af = asdf.AsdfFile({'arrs': arrs})
af.write_to(b)

b.seek(0)
with asdf.open(b) as raf:
    arefs = raf['arrs']

print(b.tell())
print(arefs[1][:])
print(b.tell())

produces:

2902
[ 1  2  3  4  5  6  7  8  9 10]
1830

showing that the array loading from the 'closed' MemoryIO resulting in a change in the file pointer position.

@braingram
Copy link
Contributor Author

This also affects opening and reading from regular files:

import asdf
import numpy

fn = 'test.asdf'
arrs = [numpy.arange(10) + i for i in range(10)]
af = asdf.AsdfFile({'arrs': arrs})
af.write_to(fn)

fp = open(fn, 'rb')
with asdf.open(fp) as raf:
    arefs = raf['arrs']

print(fp.tell())
print(arefs[1][:])
print(fp.tell())
2902
[ 1  2  3  4  5  6  7  8  9 10]
1830

@braingram
Copy link
Contributor Author

Fixed in #1537 both examples above now no longer seek the file pointer and raise an exception:

OSError: Attempt to load block from closed file

braingram added a commit to braingram/jwst that referenced this issue Oct 6, 2023
asdf previously had a bug where contents from a closed
file could be accessed. See:
asdf-format/asdf#1539

The cube build step was relying on this bug, closing models
before the wcs was fully read (this is lazy loaded).

This commit prevents the early model closure.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant