Strange failure for RDataFrame.Snapshot for the *ancient* input ROOT file

[VanyaBelyaev]

Check duplicate issues.

• Checked for duplicates

Description

I need to process ROOT file, created with some relatively old ROOT version.

and I see that wih recent ROOT versions RDataFRame.Snapshot fails to process the file, while
for old versions of ROOT it succeds

OK:

( source /cvmfs/sft.cern.ch/lcg/views/LCG_107/x86_64-el9-gcc13-opt/setup.sh ; root.exe --version ; python ./tst_frame.py )

FAILURE:

 ( source /cvmfs/sft.cern.ch/lcg/views/LCG_108/x86_64-el9-gcc13-opt/setup.sh ; root.exe --version ; python ./tst_frame.py ) 

It reports:

ROOT Version: 6.36.02
Built for linuxx8664gcc on Jul 10 2025, 20:02:19
From tags/6-36-02@6-36-02
Error in <TTreeReaderValue::Get()>: Value reader for branch m34CPVMJ not properly initialized, did you call TTreeReader::Set(Next)Entry() or TTreeReader::Next()?
RDataFrame::Run: event loop was interrupted
Error in <TTreeReaderValue::Get()>: Value reader for branch m34CPVMJ not properly initialized, did you call TTreeReader::Set(Next)Entry() or TTreeReader::Next()?
RDataFrame::Run: event loop was interrupted
Error in <TTreeReaderValue::Get()>: Value reader for branch m34CPVMJ not properly initialized, did you call TTreeReader::Set(Next)Entry() or TTreeReader::Next()?
RDataFrame::Run: event loop was interrupted
Traceback (most recent call last):
  File "/afs/cern.ch/user/i/ibelyaev/cmtuser/TEST/./tst_frame.py", line 18, in <module>
    frame.Snapshot( 'MC2' , output , ( 'm34CPVMJ' ,  'm234CPVMJ' , 'tmva_BDTG3' ) ) 
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Template method resolution failed:
  none of the 3 overloaded methods succeeded. Full details:
  ROOT::RDF::RResultPtr<ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager,void> > ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager,void>::Snapshot(string_view treename, string_view filename, initializer_list<string> columnList, const ROOT::RDF::RSnapshotOptions& options = RSnapshotOptions()) =>
    out_of_range: RDataFrame: Action (Snapshot) could not retrieve value for column 'm34CPVMJ' for entry 0. You can use the DefaultValueFor operation to provide a default value, or FilterAvailable/FilterMissing to discard/keep entries with missing values instead.
  ROOT::RDF::RResultPtr<ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager,void> > ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager,void>::Snapshot(string_view treename, string_view filename, const ROOT::RDF::ColumnNames_t& columnList, const ROOT::RDF::RSnapshotOptions& options = RSnapshotOptions()) =>
    out_of_range: RDataFrame: Action (Snapshot) could not retrieve value for column 'm34CPVMJ' for entry 0. You can use the DefaultValueFor operation to provide a default value, or FilterAvailable/FilterMissing to discard/keep entries with missing values instead.
  bad argument type for built-in operation
  ROOT::RDF::RResultPtr<ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager,void> > ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager,void>::Snapshot(string_view treename, string_view filename, initializer_list<string> columnList, const ROOT::RDF::RSnapshotOptions& options = RSnapshotOptions()) =>
    out_of_range: RDataFrame: Action (Snapshot) could not retrieve value for column 'm34CPVMJ' for entry 0. You can use the DefaultValueFor operation to provide a default value, or FilterAvailable/FilterMissing to discard/keep entries with missing values instead.
i

There are no problem with explicit TTree/TChain manipulations from this file, Only RDataFrame fails

Reproducer

The reproducer script tst_frame.py is rather trivial:

import ROOT

ROOT.ROOT.EnableImplicitMT( True ) 

## Input data

input  =  '/afs/cern.ch/work/e/enekraso/public/Bu2jpsikkpi/jkkpi/tmva/chains/chain_tmva_mc_tmva.root'
output =  'new_file.root'

chain = ROOT.TChain ( 'MC' )
chain.Add ( input  )

## use shapshot

frame  = ROOT.ROOT.RDataFrame ( chain )
output = 'new_file.root'

## FAILS!! 
frame.Snapshot( 'MC2' , output , ( 'm34CPVMJ' ,  'm234CPVMJ' , 'tmva_BDTG3' ) ) 

new_chain = ROOT.TChain ( 'MC2' )
new_chain.Add  ( output     )
new_chain.Draw ( 'm34CPVMJ' )

For simplicity it is also
linked here

If refers to the input ROOT file from public /afs/ directory, hopefully it is readable,

Input ROOT file is created with some very old ROOT verison (~5-7 years old)

( source /cvmfs/sft.cern.ch/lcg/views/LCG_107/x86_64-el9-gcc13-opt/setup.sh ; root.exe --version ; python ./tst_frame.py ; rm new_file.root ) 
( source /cvmfs/sft.cern.ch/lcg/views/LCG_108/x86_64-el9-gcc13-opt/setup.sh ; root.exe --version ; python ./tst_frame.py ; rm new_file.root ) 

ROOT version

problem occurs between 6.34 (OK) and 6.36 (FAILURE)

Installation method

cvmfs.cern.ch

Operating system

Linux, el9

Additional context

No response

[ferdymercury]

Thanks for the report. I confirm the regression. It also fails with master-debug-build, whereas it works with 6.34.02.

C++ reproducer:

#include "TChain.h"
#include "ROOT/RDataFrame.hxx"

void tst_frame() {
ROOT::EnableImplicitMT(true);
auto chain = TChain("MC");
chain.Add("/tmp/chain_tmva_mc_tmva.root");
auto frame = ROOT::RDataFrame(chain);
auto output = "new_file.root";
frame.Snapshot("MC2", output, {"m34CPVMJ", "m234CPVMJ", "tmva_BDTG3"});
auto new_chain = TChain("MC2");
new_chain.Add(output);
new_chain.Draw("m34CPVMJ");
}

Note that if one comments EnableImplicitMT, then it works fine! So it might be a missing lock?

[vepadulano]

Hi @VanyaBelyaev ,

Thank you for your report, I will take a look. Meanwhile, would you also have the code used to produce that TTree initially?

@ferdymercury thank you for the simplified reproducer, it is absolutely helpful!

[VanyaBelyaev]

Dear @vepadulano
Unfortunately I have no access to the code that produced this file.
only know that his was an ancient ROOT version dated around 2018-1019
I've intensively & successfully worked with this file (and many other files with the same production)
but I've not used RDataFrame machinery for this files. Normal TTree.Draw , TTree.Project and related TTree methods
are perfecty OK. And I've met the problem migrating my analysis from bare TTrees to to RDataFrames

[ferdymercury]

Meanwhile, would you also have the code used to produce that TTree initially?

Here a self-contained reproducer including file generator.

It works with nthreads 0 and fails with nthreads == 1, but works again with nthreads = 2, 3, 4, 5... funny. With ROOT 6.34 it works fine with nthreads = 1 (and all others).

#include "TFile.h"
#include "TTree.h"
#include "TChain.h"
#include "TSystem.h"
#include "ROOT/RDataFrame.hxx"

void tst_frame(int nthreads = 0)
{
   auto filename = "simple20164.root";
   {
      TFile f(filename, "RECREATE");
      TTree t("MC", "");
      int v1 = 1, v2 = 2, v3 = 3;
      t.Branch("m34CPVMJ", &v1);
      t.Branch("m234CPVMJ", &v2);
      t.Branch("tmva_BDTG3", &v3);
      t.Fill();
      t.Fill();
      f.Write();
   }
   {
      ROOT::EnableImplicitMT(nthreads);
      auto chain = TChain("MC");
      chain.Add(filename);
      auto frame = ROOT::RDataFrame(chain);
      auto output = "new_file.root";
      frame.Snapshot("MC2", output, {"m34CPVMJ", "m234CPVMJ", "tmva_BDTG3"});
      auto new_chain = TChain("MC2");
      new_chain.Add(output);
      new_chain.Draw("m34CPVMJ");
   }
   gSystem->Unlink("new_file.root");
   gSystem->Unlink(filename);
}

[vepadulano]

The issue is within the TTree data source of RDataFrame, that needs to act differently depending on whether implicit MT is active or not. The reason why nthreads matters is that there was a faulty logic that checked for the number of active slots rather. This is also why EnableImplicitMT(True) triggers this issue, because True is converted to 1. I opened a PR with the fix.

In general if you just pass EnableImplicitMT() or any other number of threads different than one it should work.