Add new SimBeamSpotHLLHC CondFormat #43186

francescobrivio · 2023-11-03T22:31:42Z

PR description:

Opening as draft for now:

Added a new SimBeamSpotHLLHC CondFormat and relative record
- Also added all the needed utilities to write/read/plot/inspect the new object
A better PR description will come soon
For now I just want to test that the updates I made to HLLHCEvtVtxGenerator did not change its behavior

Any feedback is welcome (@mmusich 😉 )

PR validation:

Code compiles and I ran successfully a few upgrade workflows.

Backport:

Not a backport, no backport needed.

francescobrivio · 2023-11-03T22:35:57Z

@cmsbuild please test

cmsbuild · 2023-11-03T22:41:57Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43186/37510

This PR adds an extra 80KB to repository

cmsbuild · 2023-11-03T22:42:18Z

A new Pull Request was created by @francescobrivio for master.

It involves the following packages:

CondCore/BeamSpotPlugins (db)
CondCore/Utilities (db)
CondFormats/BeamSpotObjects (alca, db)
CondFormats/DataRecord (alca, db)
CondTools/BeamSpot (alca, db)
IOMC/EventVertexGenerators (simulation)

@perrotta, @civanch, @francescobrivio, @saumyaphor4252, @consuegs, @mdhildreth can you please review it and eventually sign? Thanks.
@seemasharmafnal, @grzanka, @missirol, @tocheng, @PonIlya, @rsreds, @mmusich, @fabiocos, @yuanchao this is something you requested to watch as well.
@antoniovilela, @sextonkennedy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2023-11-04T00:18:40Z

-1

Failed Tests: UnitTests RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b23df1/35591/summary.html
COMMIT: 9cff857
CMSSW: CMSSW_13_3_X_2023-11-03-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b23df1/35591/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b23df1/35591/git-merge-result

Unit Tests

I found 1 errors in the following unit tests:

---> test testTauEmbeddingProducers had ERRORS

RelVals

136.7611

----- Begin Fatal Exception 04-Nov-2023 00:30:42 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_PAT_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoBTag/Combined/data/ParticleNetFromMiniAODAK4/CHS/Central/modelfile/model.onnx anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/136.7611_RunJetHT2016EreMINIAOD


At:
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Types.py(881): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(381): insertContentsInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(516): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Modules.py(161): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1216): _insertManyInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1490): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

136.8311

----- Begin Fatal Exception 04-Nov-2023 00:30:43 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_PAT_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoBTag/Combined/data/ParticleNetFromMiniAODAK4/CHS/Central/modelfile/model.onnx anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/136.8311_RunJetHT2017FreMINIAOD


At:
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Types.py(881): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(381): insertContentsInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(516): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Modules.py(161): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1216): _insertManyInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1490): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

136.88811

----- Begin Fatal Exception 04-Nov-2023 00:30:44 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_PAT_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoBTag/Combined/data/ParticleNetFromMiniAODAK4/CHS/Central/modelfile/model.onnx anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/136.88811_RunJetHT2018DreMINIAODUL


At:
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Types.py(881): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(381): insertContentsInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(516): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Modules.py(161): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1216): _insertManyInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1490): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

RelVals-INPUT

4.64.6_MinimumBias2010A/step2_MinimumBias2010A.log
136.72411136.72411_RunJetHT2016B_reminiaodUL/step2_RunJetHT2016B_reminiaodUL.log
136.72412136.72412_RunJetHT2016B_reminiaodUL/step2_RunJetHT2016B_reminiaodUL.log

Expand to see more relval errors ...

francescobrivio · 2023-11-05T16:00:02Z

@cmsbuild please test

cmsbuild · 2023-11-05T18:49:20Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b23df1/35613/summary.html
COMMIT: 9cff857
CMSSW: CMSSW_13_3_X_2023-11-05-0000/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/43186/35613/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 361 lines from the logs
Reco comparison results: 134 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3363010
DQMHistoTests: Total failures: 1787
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3361201
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 214 log files, 167 edm output root files, 50 DQM output files
TriggerResults: no differences found

srimanob · 2023-12-04T14:13:33Z

Sorry for late review.
I think PR is in a good shape. Could you please elaborate more on the plan? (or point to the discussion). Thx.

perrotta · 2024-01-24T09:14:44Z

@francescobrivio what is the status of the discussion on this PR? Should we find a forum for it (AlCa meeting, or SIM meeting as it was proposed before the Holidays)?

cmsbuild · 2024-02-06T10:10:24Z

Milestone for this pull request has been moved to CMSSW_14_1_X. Please open a backport if it should also go in to CMSSW_14_0_X.

IOMC/EventVertexGenerators/interface/HLLHCEvtVtxGenerator.h

cmsbuild · 2024-05-23T18:12:38Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43186/40323

ERROR: Unable to merge PR.

See log https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43186/40323/cms-checkout-topic.log

francescobrivio · 2024-05-23T18:18:53Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43186/40323

ERROR: Unable to merge PR.

See log https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43186/40323/cms-checkout-topic.log

That's understood, I will rebase after the discussion tomorrow at the Simulation meeting.

Here I just spotted a typo and I fixed from the online GH interface, so I won't forget it when I rebase.

francescobrivio added 6 commits November 3, 2023 23:18

Add new SimBeamSpotHLLHCObjects CondFormat

a36fe12

define new SimBeamSpotHLLHCObjectsRcd

5b112c2

add SimBeamSpotHLLHCObjects to payload utilities

711ac7b

add write and read plugins for SimBeamSpotHLLHCObjects and unitTest

188597a

add SimBeamSpotHLLHC PayloadInspector

26730cd

update HLLHCEvtVtxGenerator to read BeamSpot from DB

9cff857

cmsbuild added this to the CMSSW_13_3_X milestone Nov 3, 2023

cmsbuild added simulation-pending alca-pending db-pending pending-signatures tests-pending orp-pending code-checks-pending tracking labels Nov 3, 2023

cmsbuild added tests-started and removed tests-pending labels Nov 3, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Nov 3, 2023

cmsbuild added tests-rejected and removed tests-started labels Nov 4, 2023

francescobrivio mentioned this pull request Nov 5, 2023

Add SimBeamSpotObjects to MC Global Tags cms-AlCaDB/AlCaTools#86

Closed

8 tasks

cmsbuild added tests-started and removed tests-rejected labels Nov 5, 2023

This was referenced Nov 14, 2023

Fix BeamSpotRcdRead_cfg and add unitTest #43265

Merged

add verbose option to BeamSpotRcdPrinter and use it in the unit tests #43291

Merged

This was referenced Nov 23, 2023

Porting Pixel Tracks to Alpaka [Not to Merge] #41117

Closed

Phase2 Trigger Primitive generation for Ecal Barrel #42633

Merged

francescobrivio mentioned this pull request Dec 13, 2023

Implement and consume a new CondFormat for the HL-LHC SimBeamSpot cms-AlCaDB/AlCaTools#95

Open

cmsbuild added simulation-approved and removed simulation-pending labels Jan 24, 2024

cmsbuild modified the milestones: CMSSW_14_0_X, CMSSW_14_1_X Feb 6, 2024

cmsbuild mentioned this pull request Mar 19, 2024

Add GEM channel and strip payload dump utilities #44460

Merged

cmsbuild mentioned this pull request May 1, 2024

switch to CLHEP/Units/SystemOfUnits to avoid numerous single character symbols in global scope #44882

Merged

This was referenced May 23, 2024

Add new dEdx calibration and estimator #45016

Open

Add new dEdx calibration condition format #45024

Merged

francescobrivio commented May 23, 2024

View reviewed changes

IOMC/EventVertexGenerators/interface/HLLHCEvtVtxGenerator.h Outdated Show resolved Hide resolved

fix typo in comment

96d30ed

cmsbuild added simulation-pending code-checks-pending and removed simulation-approved code-checks-approved labels May 23, 2024

cmsbuild added code-checks-rejected and removed code-checks-pending labels May 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add new SimBeamSpotHLLHC CondFormat #43186

Add new SimBeamSpotHLLHC CondFormat #43186

francescobrivio commented Nov 3, 2023

francescobrivio commented Nov 3, 2023

cmsbuild commented Nov 3, 2023

cmsbuild commented Nov 3, 2023 •

edited

cmsbuild commented Nov 4, 2023

francescobrivio commented Nov 5, 2023

cmsbuild commented Nov 5, 2023

srimanob commented Dec 4, 2023

perrotta commented Jan 24, 2024

cmsbuild commented Feb 6, 2024

cmsbuild commented May 23, 2024

francescobrivio commented May 23, 2024

Add new SimBeamSpotHLLHC CondFormat #43186

Are you sure you want to change the base?

Add new SimBeamSpotHLLHC CondFormat #43186

Conversation

francescobrivio commented Nov 3, 2023

PR description:

PR validation:

Backport:

francescobrivio commented Nov 3, 2023

cmsbuild commented Nov 3, 2023

cmsbuild commented Nov 3, 2023 • edited

cmsbuild commented Nov 4, 2023

Unit Tests

RelVals

RelVals-INPUT

francescobrivio commented Nov 5, 2023

cmsbuild commented Nov 5, 2023

Comparison Summary

srimanob commented Dec 4, 2023

perrotta commented Jan 24, 2024

cmsbuild commented Feb 6, 2024

cmsbuild commented May 23, 2024

francescobrivio commented May 23, 2024

cmsbuild commented Nov 3, 2023 •

edited