Multiple RelVals: Unknown trigger name exception

[iarspider]

RelVal 1042.0 is failing in multiple IBs with the following exception:

----- Begin Fatal Exception 02-May-2024 13:41:35 CEST-----------------------
An exception of category 'Configuration' occurred while
   [0] Processing  Event run: 305064 lumi: 38 event: 58033402 stream: 0
   [1] Running path 'ALCARECOStreamPPSCalTrackBasedSelOutPath'
   [2] Calling OutputModule prePrefetchSelection()
Exception Message:
EventSelector::init, An OutputModule is using SelectEvents
to request a trigger name that does not exist
The unknown trigger name is: pathALCARECOPPSCalTrackBasedSel
----- End Fatal Exception -------------------------------------------------

full log

[iarspider]

assign alca

[cmsbuild]

New categories assigned: alca

@saumyaphor4252,@perrotta,@consuegs you have been requested to review this Pull request/Issue and eventually sign? Thanks

[cmsbuild]

cms-bot internal usage

[cmsbuild]

A new Issue was created by @iarspider.

@smuzaffar, @rappoccio, @makortel, @antoniovilela, @Dr15Jones, @sextonkennedy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[perrotta]

Also RelVal 1041 is failing in CMSSW_14_1_X_2024-05-02-1100 with the same exception. And RelVal 1040.1 is failing with the exception

----- Begin Fatal Exception 02-May-2024 09:43:06 CEST-----------------------
An exception of category 'Configuration' occurred while
   [0] Processing  Event run: 305064 lumi: 38 event: 58036426 stream: 2
   [1] Running path 'ALCARECOStreamSiPixelCalZeroBiasOutPath'
   [2] Calling OutputModule prePrefetchSelection()
Exception Message:
EventSelector::init, An OutputModule is using SelectEvents
to request a trigger name that does not exist
The unknown trigger name is: pathALCARECOSiPixelCalZeroBias
----- End Fatal Exception -------------------------------------------------

I suppose the issue is surfaced by the merging in that IB of #44767 , which throws a clear exception if OutputModule SelectEvents refers to a non-existent Path or Process.

The three failing workflows refer to 'RunExpressPhy2017F' (ExpressPhysics dataset for 2017).

As far as I can read from https://twiki.cern.ch/twiki/bin/viewauth/CMS/AlCaRecoMatrix, in Run2 there was not such a PPSCalTrackBasedSel path in ALCARECO, and in 2017 not even a SiPixelCalZeroBias. (To be honest, if I read the content of Configuration/AlCa/doc/pastAlCaRecoMatrix.py I get a different content: this is something that must be investigated further).

wf 1040.1 was introduced in #22455: my impression is that a configuration for 2018 was (wrongly) used for a 2017 AlCaReco: unfortunately, the logs of the tests cannot be recovered to verify whether any message was issued in them for the missing path.

The same for wf 1041.0 that was introduced (as previous 1040.2) with #33215, and wf 1042.0 that was introduced with #35631, but in both cases still with a configuration that does not seem to me to reflect the 2017 ALCARECO content.

This is mostly to tell that I am investigating. Let me add here @francescobrivio (as author of some of the PRs that considered the Express content for 2017), my fellow @cms-sw/alca-l2 , and @cms-sw/ctpps-dpg-l2 for a clarification about the PPS content in Express for the 2017.

And let me also add @makortel as author of #44767: did you notice Matti that after the CMSSW_14_1_X_2024-05-02-1100 the exception is not 'clear' any more (see the very same workflows failing in CMSSW_14_1 2024-05-02-2300)?. Do you understand why?

[Dr15Jones]

And let me also add @makortel as author of #44767: did you notice Matti that after the CMSSW_14_1_X_2024-05-02-1100 the exception is not 'clear' any more (see the very same workflows failing in CMSSW_14_1 2024-05-02-2300)?. Do you understand why?

We discovered several bugs in #44767 which are now fixed in #44891. These bugs resulted in the assert sometimes going off rather than the exception message being printed. We did extensive testing with the new fix and now see the exception message being issued consistently.

[makortel]

Let's use this issue to report all the "unknown trigger names" from #44891 (comment)

Workflow 1001.3

----- Begin Fatal Exception 03-May-2024 02:16:12 CEST-----------------------
An exception of category 'Configuration' occurred while
   [0] Processing  Event run: 355769 lumi: 35 event: 22779366 stream: 0
   [1] Running path 'ALCARECOStreamEcalESAlignOutPath'
   [2] Calling OutputModule prePrefetchSelection()
Exception Message:
EventSelector::init, An OutputModule is using SelectEvents
to request a trigger name that does not exist
The unknown trigger name is: pathALCARECOEcalESAlign
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 03-May-2024 02:16:12 CEST-----------------------
An exception of category 'Configuration' occurred while
   [0] Processing  Event run: 355769 lumi: 35 event: 22311221 stream: 3
   [1] Running path 'ALCARECOStreamEcalUncalZElectronOutPath'
   [2] Calling OutputModule prePrefetchSelection()
Exception Message:
EventSelector::init, An OutputModule is using SelectEvents
to request a trigger name that does not exist
The unknown trigger name is: pathALCARECOEcalUncalZElectron
----- End Fatal Exception -------------------------------------------------

1040.1

----- Begin Fatal Exception 03-May-2024 01:53:50 CEST-----------------------
An exception of category 'Configuration' occurred while
   [0] Processing  Event run: 305064 lumi: 38 event: 58036426 stream: 1
   [1] Running path 'ALCARECOStreamSiPixelCalZeroBiasOutPath'
   [2] Calling OutputModule prePrefetchSelection()
Exception Message:
EventSelector::init, An OutputModule is using SelectEvents
to request a trigger name that does not exist
The unknown trigger name is: pathALCARECOSiPixelCalZeroBias
----- End Fatal Exception -------------------------------------------------

1041.0, 1042.0

----- Begin Fatal Exception 03-May-2024 01:53:35 CEST-----------------------
An exception of category 'Configuration' occurred while
   [0] Processing  Event run: 305064 lumi: 38 event: 58067786 stream: 2
   [1] Running path 'ALCARECOStreamPPSCalTrackBasedSelOutPath'
   [2] Calling OutputModule prePrefetchSelection()
Exception Message:
EventSelector::init, An OutputModule is using SelectEvents
to request a trigger name that does not exist
The unknown trigger name is: pathALCARECOPPSCalTrackBasedSel
----- End Fatal Exception -------------------------------------------------

[makortel]

And for reference, an earlier such issue #44780 was fixed in #44787

[makortel]

My private test of 1001.3 on a single thread gives also

----- Begin Fatal Exception 02-May-2024 21:13:44 CEST-----------------------
An exception of category 'Configuration' occurred while
   [0] Processing  Event run: 355769 lumi: 35 event: 22311221 stream: 0
   [1] Running path 'ALCARECOStreamTkAlV0sOutPath'
   [2] Calling OutputModule prePrefetchSelection()
Exception Message:
EventSelector::init, An OutputModule is using SelectEvents
to request a trigger name that does not exist
The unknown trigger name is: pathALCARECOTkAlK0s
----- End Fatal Exception -------------------------------------------------

so all these workflows might have more issues than reported so far.

[mmusich]

#44914 should fix.