DPL Analysis: provide support for metadata

[ktf]

Metadata is read from the AOD files metaData tree and provided in the form of a string to any task which subscribes to it via the "Metadata" specification in the task description itself.

E.g. to know if a given task was created for run2 or run 3 you can use:

struct MyTask : AnalysisTask {
  Metadata creationRun{"Run"};

  void process(...) {
    LOGP("Run was {}", *creationRun);
  }
};

[ktf]

@ddobrigk @jgrosseo @pzhristov this provides some runtime metadata support. We should discuss the exact mechanics that we want (e.g. what if parent files provide metadata as well? which one wins?) and if string based metadata is good enough or we want something more.

[jgrosseo]

Great news!
To your questions:

• First file in the input list wins, no need to re-read, neither to go to the parent.
• I think strings are OK. Or expose the map.
  Is this already available in init. This would be great...

[ktf]

Ok, I am now working on getting the same information available at topology building time. I actually wonder if that's not all we need. Is there any case in which re-reading per file is a good idea?

[ktf]

https://github.com/AliceO2Group/AliceO2/pull/11875/files provides support at workflow construction. Maybe it's all we need...

[ddobrigk]

So about availability at init time: this would be very useful but it could also pose a problem. What if an analysis loops over many files and then we switch runs during the execution? It could be that the decisions taken at init time are then no longer appropriate for the files analysed in sequence... Or maybe this won't happen? Not sure; I guess it depends on if metadata varies run-by-run. Also: when analysing metadata, could we even switch periods while using the same analysis task? I suppose not.

[aalkin]

How does this work with a combined workflow? Would it read the AOD file each time a new workflow is added?

[aalkin]

@ddobrigk I do not believe we have a case of analyzing Run2 and Run3 data in one go. And different periods also would be analyzed separately, at least at the first filtering stage. The derived data can combine several periods probably, and thus metadata of the derived AODs should account for that.

[jgrosseo]

So about availability at init time: this would be very useful but it could also pose a problem. What if an analysis loops over many files and then we switch runs during the execution? It could be that the decisions taken at init time are then no longer appropriate for the files analysed in sequence... Or maybe this won't happen? Not sure; I guess it depends on if metadata varies run-by-run. Also: when analysing metadata, could we even switch periods while using the same analysis task? I suppose not.

Hi David,

The concept of meta data was always that it is user responsability to not mix files with different meta data content. We want to configure the topology with meta data and once done, this cannot be changed. So I would advocate to not support the use case of following the meta data file by file.

Cheers,
Jan

[alibuild]

Error while checking build/O2/fullCI for 5c34858 at 2024-01-19 04:28:

## sw/BUILD/O2-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
/sw/slc8_x86-64/fmt/10.1.1-10/include/fmt/core.h:1576:63: error: 'fmt::v10::detail::type_is_unformattable_for<TString, char> _' has incomplete type
/sw/slc8_x86-64/fmt/10.1.1-10/include/fmt/core.h:1580:7: error: static assertion failed: Cannot format an argument. To make type T formattable provide a formatter<T> specialization: https://fmt.dev/latest/api.html#udt
ninja: build stopped: subcommand failed.

Full log here.

[github-actions]

This PR did not have any update in the last 30 days. Is it still needed? Unless further action in will be closed in 5 days.