Is it possible to specify which TTree version to use when writing a new tree?

[denehoffman]

I'm trying to read in data from an existing TTree, reformat/skim it down, and write it to a new TTree. The only problem is that my entire collaboration uses an outdated version of ROOT which apparently can't read version-20 TTrees:

Error in <TBufferFile::ReadClassBuffer>: Could not find the StreamerInfo for version 20 of the class TTree, object skipped at offset 51
Error in <TBufferFile::CheckByteCount>: object of class TTree read too few bytes: 2 instead of 5528

The tree can be opened (in ROOT) just fine on my local computer running the latest ROOT. When I look at the input tree versions, I see that

>>> uproot.open("tree_sum_AMO_30274_31057.root")['ksks__B4_Tree'].__class__
<class 'uproot.models.TTree.Model_TTree_v19'>

so version-19 trees are fine, but the output trees have

>>> uproot.open("flattree_AMO.root")['kin'].__class__
<class 'uproot.models.TTree.Model_TTree_v20'>

and these ones can't be read by the collaboration's version of ROOT. Is there some way to specify using the previous version when writing?

[jpivarski]

Version 20 of TTree is hard-coded in Uproot's writer (had to pick something), and while it would be possible to add code to skip some fields and make it version 19, that feels complicate the code, not to mention the interface (you'd have to specify the version somehow).

But the error message is suggesting something worse: old versions of ROOT are supposed to be able to read new data because every file with a TTree in it should have a TStreamerInfo for that TTree version (same for all other classes). In the file that you've written, does

f.file.show_streamers()

not include TTree version 20? If it doesn't, that's bad news.

[denehoffman]

It actually doesn't include anything, in fact, it didn't return any text at all. I looked back at my code and realized I had never closed the file, so I changed a output_file = uproot.recreate(str(output_file_path)) to with uproot.recreate(str(output_file_path)) as output_file: and indented all the code below it properly, but I still get trees that can only be read on ROOT version 6.25/01 and not my cluster's ROOT version 6.08/06.

Here's the minimal code I've been able to replicate this with:

import numpy as np
import uproot

with uproot.recreate("demo.root") as output_file:
    output_file.mktree("kin", {"Weight": "f4", "Energy": ("f4", (3,))})
    for i in range(10):
        weights = np.random.rand(10000)
        energies = np.random.rand(10000, 3)
        output_file["kin"].extend({"Weight": weights, "Energy": energies})

with uproot.open("demo.root") as input_file:
    input_file.file.show_streamers()

demo.root

Let me know what I'm doing wrong here, I'm sure I made some silly mistake!

[jpivarski]

Thanks; it looks like it's not writing any streamers at all, which is not intended. I'll convert this Discussion into an Issue.

Using context managers (with blocks) is the right way to do it and it ensures that the files get properly closed, but Uproot doesn't wait for file-closure to invoke the streamer-writing process. In fact (whether this is a good idea or not), after every assignment, it calls flush to ensure that the file-updates are on their way. (flush doesn't guarantee that it writes to disk, but I think it gets all of those file-changes to the operating system, so that they're sure to be written even if the Python process is closed.) Nevertheless, using context managers is the safest you can be.

[denehoffman]

Sounds good. Out of curiosity, do you know why I'm able to read these files fine on a newer version of ROOT but not the older version? Does the newer version just have some expectation of the streamer info which happens to match what was written?

[jpivarski]

Yeah, the streamer is built into ROOT as reflection info in TClass (I assume). Starting in Uproot 4, Uproot has a few class + version combinations built in, so that it can avoid reading streamers in typical cases, but not nearly as many as ROOT. This sort of thing could be put into a giant database, rather than encoding it as code, but I haven't undertaken that project.

The fact that ROOT (and now Uproot) can read some class + version combinations without having to resort to streamers unfortunately masks situations like this, in which the streamers are actually missing.