[SimWF] Update reco and sources

[benedikt-voelkel]

• always run all digi stages independent of which detectors produce hits

• update TOF match sources

• add TRD and HMP sources

• introduces implicit modularisation

  • empty digit and reco files are always produced for missing detector
    hit files
    --> technicannly allows for any detector composition
    --> in particular crucial for anchored simulations
    --> enables the simulation of single detectors for specific expert
    studies when other material budget is not needed or has to be
    avoided even

• do not hard-code ALIEN_JDL_WORKFLOWDETECTORS but actually allow to
  take it from the outside JDL in anchorMC.sh

[benedikt-voelkel]

The initial motivation for this PR was to enable (disable) TOF when present (not present).
Since there was also some logic for TRD and there are matching sources for TOF and TRD, also took care of TRD here.

Additional detector logic will be added - if needed - after this PR.

[benedikt-voelkel]

Hello @shahor02
when I run

export ALIEN_JDL_LPMANCHORPASSNAME=apass2
export ALIEN_JDL_MCANCHOR=apass2
export ALIEN_JDL_COLLISIONSYSTEM=Pb-Pb
export ALIEN_JDL_CPULIMIT=8
export ALIEN_JDL_LPMPASSNAME=apass2
#export ALIEN_JDL_LPMRUNNUMBER=544121
export ALIEN_JDL_LPMRUNNUMBER=544091
export ALIEN_JDL_LPMPRODUCTIONTYPE=MC
export ALIEN_JDL_LPMINTERACTIONTYPE=PbPb
export ALIEN_JDL_LPMPRODUCTIONTAG=LHC24a1
#export ALIEN_JDL_LPMANCHORRUN=544121
export ALIEN_JDL_LPMANCHORRUN=544091
export ALIEN_JDL_LPMANCHORPRODUCTION=LHC22o
export ALIEN_JDL_LPMANCHORYEAR=2023

export NTIMEFRAMES=2
export NSIGEVENTS=2
export SPLITID=100
export PRODSPLIT=153
export CYCLE=0

# on the GRID, this is set, for our use case, we can mimic any job ID
export ALIEN_PROC_ID=2963436952

# run the central anchor steering script; this includes
# * derive timestamp
# * derive interaction rate
# * extract and prepare configurations (which detectors are contained in the run etc.)
# * run the simulation (and QC)
${O2DPG_ROOT}/MC/run/ANCHOR/anchorMC.sh

this PR with AliceO2Group/AliceO2#12686, I see TOF being included in the WORKFLOW_DETECTORS although that should not be there (see also Francesco's comment here https://its.cern.ch/jira/browse/O2-4615).
Do you know why that happens?

Let me know if you need more details of course.

[benedikt-voelkel]

@shahor02 With this I can make it work. Does that make sense?
Now, only the used detectors of this run are used and then setenv.sh puts things together correctly.

[shahor02]

@benedikt-voelkel surely it will work but it redefines the WORKFLOW_DETECTORS set from the JDL. It is up to DPG (@chiarazampolli @noferini ) to decide if they want to have a variable detectors list going into the reconstruction in the same period. This may alter the output files from reco which in turn may affect some postprocessing / merging etc.
The alternative could be to deactivate in the MC detectors present in the WORKFLOW_DETECTORS and absent in the RUN_DETECTOR_LIST

[benedikt-voelkel]

So, after another discussion with @shahor02 and @chiarazampolli I ended up implementing the following approach:

The WORKFLOW_DETECTORS are extracted as before, namely from

activeDetectors = anchorConfig.get('o2-ctf-reader-workflow-options').get('onlyDet','all')

These are by default all detectors or the list that has been passed to AILEN_JDL_WORKFLOWDETECTORS.

Now, with the work from @sawenzel (#1452), we in addition pull the list of actually used detectors in data taking from CCDB.

I then build the intersection of those lists, hence respecting the choice from a user steering the simulation (ALIEN_JDL_WORKFLOWDETECTORS) as well as respecting the detectors that were used in data taking.

This is then the detector list that will be passed to the transport and only those detectors will produce hit files.
Following that however, everything is digitised and reconstructed, leading to empty files in case certain detectors did not produce hits.

Indeed, that might introduce some computational overhead, on the other hand it circumvents the following problem:
In case additional options/arguments for detectors are introduced for certain digi/reco DPLs, that would make one detector DPL depend on another detector DPL (such as the option --use-ft0 in case of o2-tpcits-match-workflow), we would not need to put any checks or protection into the simulation workflow creation.

[benedikt-voelkel]

@noferini for run 544091 I now get the detectors TRD MCH CPV FT0 CTP PHS TPC MFT MID HMP ITS FV0 FDD for transport and hence only those digits/tracks and so forth.

[noferini]

@noferini for run 544091 I now get the detectors TRD MCH CPV FT0 CTP PHS TPC MFT MID HMP ITS FV0 FDD for transport and hence only those digits/tracks and so forth.

Hi @benedikt-voelkel , I checked in the RCT and the list is this one
CPV,FDD,FT0,FV0,HMP,ITS,MCH,MFT,MID,PHS,TPC,TRD,ZDC
You have CTP (but we didn't trace it in RCT so it is fine) and you don't have ZDC (it is disable by default in sim)
Therefore, it seems to me that it is what we expect. Thanks

[benedikt-voelkel]

Hi @benedikt-voelkel , I checked in the RCT and the list is this one CPV,FDD,FT0,FV0,HMP,ITS,MCH,MFT,MID,PHS,TPC,TRD,ZDC You have CTP (but we didn't trace it in RCT so it is fine) and you don't have ZDC (it is disable by default in sim) Therefore, it seems to me that it is what we expect. Thanks

Exactly, since ZDC is not in our list (see https://github.com/AliceO2Group/O2DPG/pull/1441/files#diff-ec62e7d374a5f942616197c7eb143f6ad3faba56988ed36c8f0f0e5b49f936e1R82), it will not be taken at all since I am building the intersection of the running detectors and ALIEN_JDL_WORKFLOWDETECTORS

[noferini]

Very good. Do you need to change something else or can we merge the PR?
If you want you can check also this run
544167
Det list: CPV,FDD,FT0,FV0,HMP,ITS,MCH,MFT,MID,PHS,TOF,TPC,ZDC

here you have TOF but not TRD.

I am not in the approver list, pinging @chiarazampolli

[benedikt-voelkel]

Let's wait for the CI and then we should be fine.

I check your suggested run (544167) and I get the same list 👌

And most importantly, the simulation (and QC) goes though.