Non uniquess of the UUID inside TFile

[chaen]

Check duplicate issues.

• Checked for duplicates

Description

A few weeks ago LHCb switched from using a custom UUIDv4 to identify ROOT file to the TUUID that comes from ROOT's TFile.

This led to observing collisions of GUID between files that are produced concomitantly on the same worker node on the grid.

Reproducer

1. Write mkuuid.cxx

#include "TUUID.h"
#include <cstdio>
int main() {
    TUUID u;
    puts(u.AsString());
    return 0;
}

2. g++ -O2 -o mkuuid mkuuid.cxx $(root-config --cflags --libs)
3. seq 50000 | xargs -P 512 -n1 ./mkuuid > uuids.txt
4. check for duplicates (might take some tuning depending on the machine

$ wc -l uuids.txt
50000 uuids.txt
$ sort uuids.txt | uniq -d
d90ba95e-3e4b-11f1-a11e-ebd58d80beef

ROOT version

all

Installation method

LCG and Conda

Operating system

Linux

Additional context

For reference, this was the previous LHCb implementation
https://gitlab.cern.ch/gaudi/Gaudi/-/blob/34daa81ac63690ebf0aff19e8cd4454f7bb4ef34/GaudiUtils/src/component/createGuidAsString.cpp
https://linux.die.net/man/3/uuid_generate_time

[chrisburr]

I have a patch, I'll open an MR once the build finishes.

[dpiparo]

Hello. Thanks @chaen @chrisburr !
The documentation of TUUID is somewhat clear about the limitations and already seems to propose a solution for the reproducer above [1].
Why is the alternative proposed in the TUUID doc not good for LHCb?

[1]
In the way this UUID is constructed, when used outside of their original concept (NCS), they are actually not Globally unique and indeed multiple distinct concurrent processes are actually likely to generate the same UUID. Technically this is because the UUID is constructed only from the node information and time information. To make a globally unique number, this needs to be combined with TProcessUUID.

[chrisburr]

The issue we see is with the UUID of a TFile where I don't think we have any control over what is used?

[chrisburr]

Reading more carefully the TProcessUUID mechanism is for something different.

It disambiguates TUUIDs within a single running process so that TRefs within that process can resolve to the right object.

LHCb's identifier is the GUID that TFile persists in the file header: TFile::TFile() calls fUUID = TUUID(), and TFile::WriteHeader() does fUUID.FillBuffer(buffer).

Prior to a few weeks ago we were using a Gaudi-generated UUID but it was decided to switch to using the ROOT provided UUID as it's always seemed redundant to have two unique identifiers.

[chrisburr]

Another option (which isn't mutally exclusive with #22016) would be to add something like static TUUID::UUID4() and make TFile use a random UUID instead of v1.

[ferdymercury]

Related discussion (https://its.cern.ch/jira/browse/ROOT-8535?jql=project%20%3D%20ROOT%20AND%20text%20~%20%22UUID%22%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC) where they suggest seeding TUUID by a TProcessUID or using std::chrono

[jblomer]

Another option (which isn't mutally exclusive with #22016) would be to add something like static TUUID::UUID4() and make TFile use a random UUID instead of v1.

I think that's the right approach. We can use TSystem::GetCryptoRandom() and don't need to introduce any new dependencies.

I'm not sure about #22016, I think I would prefer a single, consistent way of creating UUIDs that doesn't depend on whether libuuid happens to be present.

[chrisburr]

Feel free to close #22016 if you'd prefer another route.

From the LHCb perspective all that is needed is that TFile::GetUUID() returns something that is actually unique.