Ecal phisym run3 workflow

[simonepigazzini]

PR description:

• Run1 EcalPhiSym code removed from Calibration/EcalCalibAlgos
• Run3 workflow added:
  • EcalPhiSymRecHit and EcalPhiSymInfo dataformats
  • Object producers and flat table producers for NanoAOD output
  • config files and test workflow
• The EcalPhiSymRecHitProducer comes in two flavors: EndRun and EndLuminosityBlock producer, a base class
  provides the common methods. The producers are edm::global, I ran a basic test to check that the products are identical
  when running with a single or multiple threads.
• It is unclear if any thread safety should be implemented using a edm::SerialTaskQueue. The documentation available here does not reflect the current interface of the class.

At the moment, both the new data formats and the producer have been placed inside Calibration/EcalCalibAlgos. The data formats might be moved somewhere else, either DataFormats/EcalRecHit or a new package under CalibFormats/. I do not have strong opinions, the current option keeps everything "close" and in the same place where the ECALPedestalPCL workflow is. Eventually, we plan to run this workflow at T0 as part of ALCARECO (pending further discussion with AlCa and T0).

[simonepigazzini]

@thomreis

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36988/28374

• This PR adds an extra 32KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-checks:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36988/28374/code-checks.patch
  e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36988/28374/code-checks.patch | patch -p1
  You can also run scram build code-checks to apply code checks directly

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36988/28375

• This PR adds an extra 32KB to repository

[cmsbuild]

A new Pull Request was created by @simonepigazzini for master.

It involves the following packages:

• Calibration/EcalCalibAlgos (alca)

@cmsbuild, @malbouis, @tvami, @yuanchao, @francescobrivio can you please review it and eventually sign? Thanks.
@rchatter, @tocheng, @argiro, @thomreis, @simonepigazzini, @mmusich this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[simonepigazzini]

@mariadalfonso this is follows #36902

[simonepigazzini]

@thomreis @francescobrivio the last commit should implement all the changes you requested.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36988/28395

• This PR adds an extra 32KB to repository

[francescobrivio]

@cmsbuild please test

[cmsbuild]

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-70bdfe/23277/summary.html
COMMIT: 5267111
CMSSW: CMSSW_12_4_X_2022-03-22-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36988/23277/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

The relvals timed out after 4 hours.

• 4.64.6_MinimumBias2010A+MinimumBias2010A+RECOSKIMALCA+HARVESTDR1/step2_MinimumBias2010A+MinimumBias2010A+RECOSKIMALCA+HARVESTDR1.log
• 136.8311136.8311_RunJetHT2017F_reminiaod+RunJetHT2017F_reminiaod+REMINIAOD_data2017+HARVEST2017_REMINIAOD_data2017/step2_RunJetHT2017F_reminiaod+RunJetHT2017F_reminiaod+REMINIAOD_data2017+HARVEST2017_REMINIAOD_data2017.log
• 136.83111136.83111_RunJetHT2017F_reminiaodUL+RunJetHT2017F_reminiaodUL+REMINIAOD_data2017UL+HARVEST2017_REMINIAOD_data2017UL/step2_RunJetHT2017F_reminiaodUL+RunJetHT2017F_reminiaodUL+REMINIAOD_data2017UL+HARVEST2017_REMINIAOD_data2017UL.log

Comparison Summary

There are some workflows for which there are errors in the baseline:
534.0 step 1
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-70bdfe/39434.911_TTbar_14TeV+2026D88_DD4hep+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal

Summary:

• No significant changes to the logs found
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3695650
• DQMHistoTests: Total failures: 52027
• DQMHistoTests: Total nulls: 61
• DQMHistoTests: Total successes: 3643540
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 204 log files, 45 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[tvami]

@cmsbuild , please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-70bdfe/23301/summary.html
COMMIT: 5267111
CMSSW: CMSSW_12_4_X_2022-03-22-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36988/23301/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

There are some workflows for which there are errors in the baseline:
534.0 step 1
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-70bdfe/39434.911_TTbar_14TeV+2026D88_DD4hep+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3695650
• DQMHistoTests: Total failures: 52021
• DQMHistoTests: Total nulls: 61
• DQMHistoTests: Total successes: 3643546
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 204 log files, 45 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[francescobrivio]

+1

• Differences don't come from this PR but from known issues Comparison failed for tests of wf 39434.911 #37315 and Spurious failing comparisons entries in PR tests for MuonCSCDigisV/CSCDigiTask/CLCT #37268
• @simonepigazzini we might need to add a relval wf testing these changes (see Ecal phisym run3 workflow #36988 (comment)), but this can be done in a follow up PR so we can merge this PR now.
• @simonepigazzini do we also need a backport to 12_3? I guess it depends on how this will be put in production, let's see how the discussion at https://indico.cern.ch/event/1137466/#b-455328-discussion-of-nano-fo goes

[simonepigazzini]

Ciao @francescobrivio ,

I think relval is a good idea, we can add them in a second PR. Backport might be good to have for testing, I need to refresh myself how to do that, it should be straight forward since the package hasn't change in a long time.

[francescobrivio]

@cms-sw/orp-l2 ok given the comments in #36988 (comment) and #36988 (comment) this PR can be merged.

[qliphy]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.