Compatibility with file format version 1.1

[philsmt]

As learned during a deployment test of the new pclayer version, it now writes an additional row in the beginning of METADATA/dataSources datasets, namely:

• An empty string in METADATA/dataSources/root
• The string Karabo_TimerServer in METADATA/dataSources/dataSourceId and METADATA/dataSources/deviceId

As the entry in METADATA/dataSources/dataSourceId contains no valid prefix (either CONTROL or INSTRUMENT), it breaks any and all reading code. This PR makes FileAccess._read_data_sources ignore this entry, checked successfully against data taken during the deployment test.

As far as I can see, we do not read data from this group anywhere else.

EDIT: Now with format version 1.2 deployed, I've changed this PR to limited compatibility with 1.1, see below.

[takluyver]

Thanks! Is this specific name hardcoded in the DAQ? There's no chance that we'd hit something similar with other names?

[takluyver]

To record the answer to my question: yes, this name is quite distinctly hardcoded into the machinery that writes the file:

https://git.xfel.eu/gitlab/karaboDevices/pcLayer/blob/1.10.3-2.10.5/src/devices/DataAggregator/Writer.cc#L1633

We're waiting to get official word of whether this is deliberate before we merge this and release.

[philsmt]

I'll keep this MR around until we see the new pclayer tag that no longer needs it.

[takluyver]

I'm wondering if we'll want to merge it anyway, so that the files created in the meantime can be opened. It should be harmless for the other files.

[philsmt]

Maybe, I did not consider the files already written.

We can hope though that it did not extend beyond some test files during deployment testing, and simply pretend this never happened. Otherwise, it might even be useful to bump the format version number again to sort out the "broken" flag dataset.

[philsmt]

I've changed the wording and added a warning when the validation flag is used. I've also added basic compatibility with 1.0-style validation by inverting the INDEX/flag dataset,.

[takluyver]

LGTM 👍

[philsmt]

Thanks for the review!

When we implement some API for the 1.2 semantics to make use of specific source to use for validation, we might be able to also apply this to 1.1, but treating it like 1.0 is probably enough.