Add a tool to merge several podio files into a single one

[jmcarcell]

BEGINRELEASENOTES

• Add a tool to merge several podio files into a single one:
  • Metadata for every event can be saved or not
  • The same format as the first input file will be used (TTree or RNTuple)
  • Metadata about the input file names will be saved

ENDRELEASENOTES

Useful not to have to deal with many small files, even though the readers can read them fine.

[tmadlener]

This does a lot of heavy lifting that might not be strictly necessary, as it unpacks and than repacks every collection in every frame, whereas a simple hadd would almost be enough for merging TTree based files. Additionally, it will do schema evolution (if applicable), so that there could be subtle changes to the produced output file. On the other hand, this will correctly handle input files with different schema versions, so that might be not too bad.

How much slower is this than a c++ implementation? There is #620 after all to solve a quite significant performance issue with podio-dump, and the difference is not only the long startup time in that case.

[jmcarcell]

Ah I didn't think about hadd, this is a tool I have used a couple of times (well only the for loop) so I thought it would be nice to have. On small files (1 MB TTree files) it's much faster than in python, but that's the case with the worst relative overhead I would imagine. For RNTuples it crashes with the ones I tried.

[tmadlener]

For RNTuples it crashes with the ones I tried.

Can you specify what crashes in this case? hadd or the c++ version of this script?

[jmcarcell]

hadd is crashing

[tmadlener]

Maybe add a bit of metadata that keeps track of which files the the merged files comes from.

[jmcarcell]

This should be ready and now write some metadata with the names of the files as passed in the arguments (full paths being saved as full paths):

Key                           Value
--------------------------------------------------------------------------------
MergeInputFiles               [example_frame.root, example_frame2.root]