Skip to content
This repository has been archived by the owner on Jun 21, 2022. It is now read-only.

uproot.open should implement support for with statement #4

Closed
vkuznet opened this issue Sep 19, 2017 · 1 comment
Closed

uproot.open should implement support for with statement #4

vkuznet opened this issue Sep 19, 2017 · 1 comment

Comments

@vkuznet
Copy link

vkuznet commented Sep 19, 2017

In python almost all open operations can be supported in with statement. You need to implement open with enter and exit method to do that. The following snippet

try:
    with uproot.open(fname) as tree:
        treeContent(tree)
except:
    traceback.print_exc()

produces the following traceback

Traceback (most recent call last):
  File "vk_test.py", line 17, in <module>
    with uproot.open(fname) as tree:
AttributeError: __exit__
@jpivarski
Copy link
Member

Of course! Sorry I overlooked that.

In #6, I added __enter__ and __exit__ methods, but they do nothing. The file in this case is not an ordinary file, but a Numpy memory-mapped array. That's because we don't want to do all of the reading between __enter__ and __exit__, but get objects on demand, as in the following example:

>>> import uproot
>>> with uproot.open("tests/Zmumu.root") as tree:
...   print tree.contents
... 
{'events;1': 'TTree'}
>>> tree["events"]
<TTree 'events' len=2304 at 0x7e564c3ca6d0>

Arguably, you could say that one should get objects from the file only within the with and at no other times. If you violated this rule with an ordinary file, you'd get a "file closed" exception and wouldn't have much trouble figuring out what went wrong. On a memory mapped file, you'd get a segmentation fault. They should only be closed when their reference count is zero.

(Memory maps are important for performance: to be able to deal with the amount of seeking the ROOT format requires and to allow parallel reading.)

On the other hand, I do have a non-memory mapped option (that I'm thinking of removing), as well as an XRootD option. I put a close() statement in __exit__ for these cases only:

>>> with uproot.open("tests/Zmumu.root", memmap=False) as tree:
...   print tree.contents
... 
{'events;1': 'TTree'}
>>> tree["events"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "uproot/rootio.py", line 149, in __getitem__
    return self.get(name)
  File "uproot/rootio.py", line 179, in get
    return self.dir.get(name, cycle)
  File "uproot/rootio.py", line 241, in get
    out = out.keys.get(n, cycle)
  File "uproot/rootio.py", line 312, in get
    return key.get()
  File "uproot/rootio.py", line 366, in get
    out = Deserialized.classes[self.classname](self._filewalker, self._walker)
  File "uproot/tree.py", line 67, in __init__
    vers, bcnt = walker.readversion()
  File "uproot/_walker/lazyarraywalker.py", line 96, in readversion
    if not self._evaluated: self._evaluate()
  File "uproot/_walker/lazyarraywalker.py", line 37, in _evaluate
    walker.startcontext()
  File "uproot/_walker/localfilewalker.py", line 61, in startcontext
    self.file.seek(self.index)
ValueError: I/O operation on closed file

Good.

flome pushed a commit to flome/uproot that referenced this issue May 11, 2020
…to-methods

Move conversion code to methods
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants