[RF] HistFactory workspaces incompatible between versions 6.26 and 6.28

[alexander-held]

Check duplicate issues.

• Checked for duplicates

Describe the bug

It is not possible to operate on a HistFactory Measurement object in a workspace created with ROOT 6.26 by using ROOT 6.28. Attempting to do so results in the following:

Error in <TRint::HandleTermInput()>: std::length_error caught: basic_string::_M_replace_aux

The opposite scenario (workspace created in 6.28, using 6.26 to read it) also causes issues. This means that there is essentially some trial&error when handling unknown workspaces to find a suitable ROOT version.

What is the expected behaviour?

Unclear to me whether this breaking change is expected or not, but the error message is not particularly clear in case this is expected.

How to reproduce?

Create HistFactory workspace in ROOT 6.26 (or use this example https://cernbox.cern.ch/s/uRaTglosZypwm81, created from xmls via hist2workspace using 6.26/08), here called workspace_example.root:

root@2cd219ab253b:~/data# root --version
ROOT Version: 6.28/04
Built for linuxx8664gcc on May 24 2023, 17:37:00
From tags/v6-28-04@v6-28-04
root@2cd219ab253b:~/data# root -l workspace_example.root
root [0]
Attaching file workspace_example.root as _file0...
(TFile *) 0x402fde6770
root [1] minimal_example->PrintXML()
Error in <TRint::HandleTermInput()>: std::length_error caught: basic_string::_M_replace_aux

ROOT version

6.28/04

How did you install ROOT?

custom build, container via https://gitlab.cern.ch/atlas-amglab/atlstats

Which operating system are you using?

N/A

Additional context

No response

[guitargeek]

Hi, thanks for reporting this!

It is indeed a bug that you can't read the 6.26 workspace in 6.28. The other way around is expected, with RooFit you can't read back a RooWorkspace with any ROOT version that is older that the version it was created with.

When I run the reproducer in the debugger, I get this stack trace:

Program received signal SIGABRT, Aborted.
0x00007ffff4a9f26c in ?? () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff4a9f26c in ?? () from /usr/lib/libc.so.6
#1  0x00007ffff4a4fa08 in raise () from /usr/lib/libc.so.6
#2  0x00007ffff4a38538 in abort () from /usr/lib/libc.so.6
#3  0x00007ffff4c9ca6f in __gnu_cxx::__verbose_terminate_handler () at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/vterminate.cc:95
#4  0x00007ffff4cb011c in __cxxabiv1::__terminate (handler=<optimized out>) at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:48
#5  0x00007ffff4cb0189 in std::terminate () at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:58
#6  0x00007ffff4cb03ed in __cxxabiv1::__cxa_throw (obj=<optimized out>, tinfo=0x7ffff4e6c0d0 <typeinfo for std::length_error>,
    dest=0x7ffff4cc8580 <std::length_error::~length_error()>) at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_throw.cc:98
#7  0x00007ffff4ca01ca in std::__throw_length_error (__s=__s@entry=0x7ffff79711aa "basic_string::_M_replace_aux")
    at /usr/src/debug/gcc/gcc/libstdc++-v3/src/c++11/functexcept.cc:82
#8  0x00007ffff76f2ae8 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_check_length (__s=<optimized out>,
    __n2=<optimized out>, __n1=<optimized out>, this=<optimized out>) at /usr/include/c++/13.1.1/bits/basic_string.h:403
#9  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_replace_aux (__c=<optimized out>, __n2=<optimized out>,
    __n1=<optimized out>, __pos1=<optimized out>, this=<optimized out>) at /usr/include/c++/13.1.1/bits/basic_string.tcc:450
#10 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::append (__c=<optimized out>, __n=<optimized out>, this=<optimized out>)
    at /usr/include/c++/13.1.1/bits/basic_string.h:1489
#11 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::resize (__c=<optimized out>, __n=<optimized out>, this=<optimized out>)
    at /usr/include/c++/13.1.1/bits/basic_string.tcc:401
#12 TBufferFile::ReadStdString (this=0x7fffffffd9d0, obj=0x7fffa8bd2d68) at /home/rembserj/spaces/master/root/src/root/io/io/src/TBufferFile.cxx:247
#13 0x00007ffff76eb1f1 in TClass::Streamer (onfile_class=0x0, b=..., obj=<optimized out>, this=0x555555cc34b0)
    at /home/rembserj/spaces/master/root/src/root/core/meta/inc/TClass.h:610
#14 TBufferFile::ReadFastArray (this=0x7fffffffd9d0, start=<optimized out>, cl=0x555555cc34b0, n=<optimized out>, streamer=<optimized out>, onFileClass=0x0)
    at /home/rembserj/spaces/master/root/src/root/io/io/src/TBufferFile.cxx:1609
#15 0x00007ffff7931e28 in TStreamerInfo::ReadBuffer<TVirtualCollectionProxy> (this=0x555558509890, b=..., arr=..., compinfo=0x5555585140d0,
    first=first@entry=0, last=4, narr=16777216, eoffset=0, arrayMode=1)
    at /home/rembserj/spaces/master/root/src/root/io/io/src/TStreamerInfoReadBuffer.cxx:1297
#16 0x00007ffff787d2bd in TStreamerInfo::ReadBufferSTL (v7=<optimized out>, eoffset=<optimized out>, nc=<optimized out>, cont=<optimized out>, b=...,
    this=<optimized out>) at /home/rembserj/spaces/master/root/src/root/io/io/src/TStreamerInfoReadBuffer.cxx:1741
#17 TStreamerInfo::ReadBufferSTL (this=<optimized out>, b=..., cont=<optimized out>, nc=<optimized out>, eoffset=<optimized out>, v7=<optimized out>)
    at /home/rembserj/spaces/master/root/src/root/io/io/src/TStreamerInfoReadBuffer.cxx:1737
#18 0x00007ffff7931b46 in TStreamerInfo::ReadBuffer<TVirtualCollectionProxy> (this=0x555556c6f630, b=..., arr=..., compinfo=0x55555864f9a0,
    first=first@entry=0, last=16, narr=2, eoffset=0, arrayMode=1) at /home/rembserj/spaces/master/root/src/root/io/io/src/TStreamerInfoReadBuffer.cxx:1274
#19 0x00007ffff787d2bd in TStreamerInfo::ReadBufferSTL (v7=<optimized out>, eoffset=<optimized out>, nc=<optimized out>, cont=<optimized out>, b=...,
    this=<optimized out>) at /home/rembserj/spaces/master/root/src/root/io/io/src/TStreamerInfoReadBuffer.cxx:1741
#20 TStreamerInfo::ReadBufferSTL (this=<optimized out>, b=..., cont=<optimized out>, nc=<optimized out>, eoffset=<optimized out>, v7=<optimized out>)
    at /home/rembserj/spaces/master/root/src/root/io/io/src/TStreamerInfoReadBuffer.cxx:1737
#21 0x00007ffff793f598 in TStreamerInfo::ReadBuffer<char**> (this=0x555558602630, b=..., arr=@0x7fffffffd740: 0x55555862ab20,
    compinfo=compinfo@entry=0x55555862ab08, first=first@entry=0, last=last@entry=1, narr=2, eoffset=0, arrayMode=3)
    at /home/rembserj/spaces/master/root/src/root/io/io/src/TStreamerInfoReadBuffer.cxx:1274
#22 0x00007ffff77f92ac in TStreamerInfoActions::VectorLooper::GenericRead (buf=..., start=<optimized out>, end=0x555556a3aaa0, loopconfig=<optimized out>,
    config=0x55555862aaf0) at /home/rembserj/spaces/master/root/src/root/io/io/src/TStreamerInfoActions.cxx:1883
#23 0x00007ffff76e978c in TStreamerInfoActions::TConfiguredAction::operator() (loopconf=0x55555862a9e0, end_collection=0x555556a3aaa0,
    start_collection=0x555556a3a850, buffer=..., this=0x55555862a9b0) at /home/rembserj/spaces/master/root/src/root/io/io/inc/TStreamerInfoActions.h:131
#24 TBufferFile::ApplySequence (this=0x7fffffffd9d0, sequence=..., start_collection=0x555556a3a850, end_collection=0x555556a3aaa0)
    at /home/rembserj/spaces/master/root/src/root/io/io/src/TBufferFile.cxx:3646
#25 0x00007ffff781f65e in TStreamerInfoActions::ReadSTLMemberWiseSameClass (buf=..., addr=<optimized out>, conf=conf@entry=0x555558626dd0,
    vers=<optimized out>) at /home/rembserj/spaces/master/root/src/root/io/io/src/TStreamerInfoActions.cxx:1155
#26 0x00007ffff781f896 in TStreamerInfoActions::ReadSTL<&TStreamerInfoActions::ReadSTLMemberWiseSameClass, &TStreamerInfoActions::ReadSTLObjectWiseFastArray>
    (buf=..., addr=0x55555855a0c0, conf=0x555558626dd0) at /home/rembserj/spaces/master/root/src/root/io/io/src/TStreamerInfoActions.cxx:1405
#27 0x00007ffff76e9c05 in TStreamerInfoActions::TConfiguredAction::operator() (object=0x55555855a0c0, buffer=..., this=0x55555812a1b0)
    at /home/rembserj/spaces/master/root/src/root/io/io/inc/TStreamerInfoActions.h:123
#28 TBufferFile::ApplySequence (this=0x7fffffffd9d0, sequence=..., obj=0x55555855a0c0)
    at /home/rembserj/spaces/master/root/src/root/io/io/src/TBufferFile.cxx:3580
#29 0x00007ffff76f1b94 in TBufferFile::ReadClassBuffer (this=0x7fffffffd9d0, cl=0x555558490b70, pointer=<optimized out>, onFileClass=<optimized out>)
    at /home/rembserj/spaces/master/root/src/root/io/io/src/TBufferFile.cxx:3499
#30 0x00007ffff778a22e in TClass::Streamer (onfile_class=0x0, b=..., obj=0x55555855a0c0, this=0x555558490b70)
    at /home/rembserj/spaces/master/root/src/root/core/meta/inc/TClass.h:610
#31 TKey::ReadObjectAny (this=0x5555569f77a0, expectedClass=<optimized out>) at /home/rembserj/spaces/master/root/src/root/io/io/src/TKey.cxx:1108
#32 0x00007ffff774aa1b in TDirectoryFile::GetObjectChecked (this=0x555556a15ce0, namecycle=<optimized out>, expectedClass=0x555558490b70)
    at /home/rembserj/spaces/master/root/src/root/io/io/src/TDirectoryFile.cxx:1111
#33 0x00005555555551f6 in TDirectory::Get<RooStats::HistFactory::Measurement> (this=0x555556a15ce0, namecycle=<optimized out>)
    at /home/rembserj/spaces/master/root/src/build/../install/include/root/TDirectory.h:207
#34 TDirectoryFile::Get<RooStats::HistFactory::Measurement> (this=0x555556a15ce0, namecycle=<optimized out>)
    at /home/rembserj/spaces/master/root/src/build/../install/include/root/TDirectoryFile.h:84
#35 repro () at repro.C:10

So the crash is here (https://github.com/root-project/root/blob/master/io/io/src/TBufferFile.cxx#L245):

      if (nwh == 255)  {
         *this >> nbig;
         obj->resize(nbig,'\0');
         ReadFastArray((char*)obj->data(),nbig);
      }

I have put some debug printouts. When it crashes, nbig is -1, and resize() expects and unsigned int. The -1 becomes a very large unsigned number that exceeds the maximum allowed length of a std::string.

To me that looks more like a problem with IO than with HistFactory (the memory layout of the HistFactory::Measurement classes hasn't changed from 6.26 to 6.28. Is that something that makes sense to you, @pcanal? What does this nbig stand for, and in which circumstances can this be -1?

Maybe in the best case, the fix would just be adding a simple if(nbig >= 0) before the call to resize().

[guitargeek]

Okay I found the fix: adding some missing entries to the LinkDef.h. This will be fixed in the next 6.28 patch release, which is 6.28.06.

[alexander-held]

Thanks a lot for looking into this! Great to hear that this will be fixed in the next release.