[PPS] Direct simulation as sequence #38280

forthommel · 2022-06-07T23:51:49Z

PR description:

This PR aims at transforming the collection of validation tools into a standard sequence for the direct simulation of strips/diamond/pixel detectors. Additionally to the moving of the CTPPSDirectProtonSimulation plugin (renamed PPSDirectProtonSimulation to follow the newest convention) from Validation/CTPPS to a dedicated SimPPS/DirectSimProducer subpackage, the configuration scripts calling this latter are also reshuffled to define a more concise, and era-based steering of the PPS direct simulation.

Consequently, this latter can now be added sequentially to a standard configuration via a standard cmsDriver customisation flag, e.g.
cmsDriver.py [...] --customise SimPPS/Configuration/Utils.setupPPSDirectSim[Mini]AOD

Additionally, a new 2022 PPS era was introduced in preparation to future conditions, both in reconstruction and in direct simulation. To ease the development of later conditions, the ctpps era is now considering run 3 detectors occupancy as its default scheme, and modifiers are only introduced for 2016-17 (with no pixels and no diamond subdetectors respectively)

PR validation:

We performed a bin-by-bin comparison of every distribution/profile introduced earlier in direct simulation validation tool and did not spot any noticeable difference.

if this PR is a backport please specify the original PR and why you need to backport that PR:

Before submitting your pull requests, make sure you followed this checklist:

verify that the PR is really intended for the chosen branch
verify that changes follow CMS Naming, Coding, And Style Rules
verify that the PR passes the basic test procedure suggested in the CMSSW PR instructions

cmsbuild · 2022-06-07T23:58:18Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38280/30430

This PR adds an extra 140KB to repository
Found files with invalid states:
- SimPPS/Configuration/python/ppsDirectSim_cff.py:
  - Added: 7fd6ca8
  - Modified: 230e9fd, 1411ff5, bd7fd49, a4a9f9d
  - Deleted: 7e0a6df
- SimPPS/DirectSimProducer/python/profile_base_2017_cff.py:
  - Added: d60dfb9
  - Deleted: d49b2f2
- SimPPS/DirectSimProducer/python/year_2017_cff.py:
  - Added: 1411ff5
  - Modified: bd7fd49, a4a9f9d
  - Deleted: d60dfb9
- SimPPS/DirectSimProducer/python/profile_2022_default_cff.py:
  - Added: 1411ff5
  - Modified: bd7fd49, d60dfb9, d49b2f2
  - Deleted: 440ef9c
- SimPPS/DirectSimProducer/python/profile_base_2021_cff.py:
  - Added: d60dfb9
  - Deleted: d49b2f2
- SimPPS/DirectSimProducer/python/profile_base_2022_cff.py:
  - Added: d60dfb9
  - Modified: d49b2f2
  - Deleted: 440ef9c
- SimPPS/DirectSimProducer/python/profile_2018_postTS2_cff.py:
  - Added: 1411ff5
  - Modified: bd7fd49, d60dfb9, d49b2f2
  - Deleted: 440ef9c
- SimPPS/DirectSimProducer/python/year_2022_cff.py:
  - Added: 1411ff5
  - Modified: a4a9f9d
  - Deleted: d60dfb9
- SimPPS/DirectSimProducer/python/year_2021_cff.py:
  - Added: 1411ff5
  - Modified: bd7fd49, a4a9f9d
  - Deleted: d60dfb9
- SimPPS/DirectSimProducer/python/profile_2018_preTS1_cff.py:
  - Added: 1411ff5
  - Modified: bd7fd49, d60dfb9, d49b2f2
  - Deleted: 440ef9c
- SimPPS/DirectSimProducer/python/profile_base_2016_cff.py:
  - Added: d60dfb9
  - Deleted: d49b2f2
- SimPPS/DirectSimProducer/python/profile_2016_preTS2_cff.py:
  - Added: 1411ff5
  - Modified: bd7fd49, d60dfb9, d49b2f2
  - Deleted: 440ef9c
- SimPPS/DirectSimProducer/python/profile_2016_postTS2_cff.py:
  - Added: 1411ff5
  - Modified: bd7fd49, d60dfb9, d49b2f2
  - Deleted: 440ef9c
- SimPPS/DirectSimProducer/python/profile_2017_preTS2_cff.py:
  - Added: 1411ff5
  - Modified: bd7fd49, d60dfb9, d49b2f2
  - Deleted: 440ef9c
- SimPPS/DirectSimProducer/python/profile_2017_postTS2_cff.py:
  - Added: 1411ff5
  - Modified: bd7fd49, d60dfb9, d49b2f2
  - Deleted: 440ef9c
- SimPPS/Configuration/test/test_miniAOD_cfg.py:
  - Added: 230e9fd
  - Modified: bd7fd49, cb5c8aa, c3f6a05, a475a6f
  - Deleted: 821e7e3
- SimPPS/DirectSimProducer/python/year_2018_cff.py:
  - Added: 1411ff5
  - Modified: bd7fd49, a4a9f9d
  - Deleted: d60dfb9
- SimPPS/DirectSimProducer/python/base_cff.py:
  - Added: 1411ff5
  - Deleted: 8528944
- SimPPS/DirectSimProducer/python/profile_2021_default_cff.py:
  - Added: 1411ff5
  - Modified: bd7fd49, d60dfb9, d49b2f2
  - Deleted: 440ef9c
- SimPPS/DirectSimProducer/python/profile_base_2018_cff.py:
  - Added: d60dfb9
  - Deleted: d49b2f2
- SimPPS/DirectSimProducer/python/year_2016_cff.py:
  - Added: 1411ff5
  - Modified: bd7fd49, a4a9f9d
  - Deleted: d60dfb9
- SimPPS/DirectSimProducer/python/profile_2018_TS1_TS2_cff.py:
  - Added: 1411ff5
  - Modified: bd7fd49, d60dfb9, d49b2f2
  - Deleted: 440ef9c
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/StandardSequences/python/Eras.py modified in PR(s): Add run3 tracking low pu era #33532

cmsbuild · 2022-06-07T23:58:43Z

A new Pull Request was created by @forthommel (Laurent Forthomme) for master.

It involves the following packages:

Configuration/Eras (operations)
Configuration/StandardSequences (operations)
Geometry/VeryForwardGeometry (geometry)
RecoPPS/Configuration (reconstruction)
RecoPPS/Local (reconstruction)
SimPPS/Configuration (simulation)
SimPPS/DirectSimProducer (****)
Validation/CTPPS (dqm)

The following packages do not have a category, yet:

SimPPS/DirectSimProducer
Please create a PR for https://github.com/cms-sw/cms-bot/blob/master/categories_map.py to assign category

@perrotta, @civanch, @Dr15Jones, @clacaputo, @makortel, @cvuosalo, @emanueleusai, @ianna, @mdhildreth, @cmsbuild, @jfernan2, @ahmad3213, @pmandrik, @slava77, @jpata, @qliphy, @rvenditti, @micsucmed, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@fabiocos, @grzanka, @missirol, @makortel, @felicepantaleo, @fabferro, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @Martin-Grunewald, @bsunanda, @lecriste, @mtosi, @ebrondol, @mmusich, @dgulhan, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

jpata · 2022-06-08T07:09:21Z

@cmsbuild please test

cmsbuild · 2022-06-08T08:28:14Z

-1

Failed Tests: RelVals RelVals-INPUT AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-70a87b/25354/summary.html
COMMIT: 818459f
CMSSW: CMSSW_12_5_X_2022-06-07-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38280/25354/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

138.4

----- Begin Fatal Exception 08-Jun-2022 09:29:10 CEST-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing global begin Run run: 346512
   [1] Calling method for module CTPPSDiamondDQMSource/'ctppsDiamondDQMOfflineSource'
Exception Message:
No "VeryForwardRealGeometryRecord" record found in the EventSetup.

 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

140.56

----- Begin Fatal Exception 08-Jun-2022 09:33:50 CEST-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing global begin Run run: 326479
   [1] Calling method for module CTPPSDiamondDQMSource/'ctppsDiamondDQMOfflineSource'
Exception Message:
No "VeryForwardRealGeometryRecord" record found in the EventSetup.

 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

1306.0

----- Begin Fatal Exception 08-Jun-2022 09:37:11 CEST-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing global begin Run run: 1
   [1] Calling method for module CTPPSDiamondDQMSource/'ctppsDiamondDQMOfflineSource'
Exception Message:
No "VeryForwardRealGeometryRecord" record found in the EventSetup.

 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

RelVals-INPUT

138.4138.4_PromptCollisions+RunMinimumBias2021+ALCARECOPROMPTR3+HARVESTDPROMPTR3/step2_PromptCollisions+RunMinimumBias2021+ALCARECOPROMPTR3+HARVESTDPROMPTR3.log
140.56140.56_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18/step2_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18.log
140.57140.57_RunHI2018Reduced+RunHI2018Reduced+RECOHID18+HARVESTDHI18/step2_RunHI2018Reduced+RunHI2018Reduced+RECOHID18+HARVESTDHI18.log

Expand to see more relval errors ...

AddOn Tests

hlt_mc_Fake2

----- Begin Fatal Exception 08-Jun-2022 09:34:04 CEST-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 5 stream: 1
   [1] Running path 'FEVTDEBUGHLToutput_step'
   [2] Prefetching for module PoolOutputModule/'FEVTDEBUGHLToutput'
   [3] Prefetching for module CTPPSLocalTrackLiteProducer/'ctppsLocalTrackLiteProducer'
   [4] Prefetching for module TotemRPLocalTrackFitter/'totemRPLocalTrackFitter'
   [5] Calling method for module TotemRPUVPatternFinder/'totemRPUVPatternFinder'
Exception Message:
No "VeryForwardRealGeometryRecord" record found in the EventSetup.

 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

hlt_data_Fake2

----- Begin Fatal Exception 08-Jun-2022 09:33:32 CEST-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing  Event run: 272762 lumi: 51 event: 41643921 stream: 1
   [1] Running path 'FEVTDEBUGHLToutput_step'
   [2] Prefetching for module PoolOutputModule/'FEVTDEBUGHLToutput'
   [3] Prefetching for module CTPPSLocalTrackLiteProducer/'ctppsLocalTrackLiteProducer'
   [4] Prefetching for module TotemRPLocalTrackFitter/'totemRPLocalTrackFitter'
   [5] Calling method for module TotemRPUVPatternFinder/'totemRPUVPatternFinder'
Exception Message:
No "VeryForwardRealGeometryRecord" record found in the EventSetup.

 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

SimPPS/Configuration/python/directSimPPS_cff.py

SimPPS/DirectSimProducer/python/profiles_2021_cff.py

SimPPS/DirectSimProducer/python/simPPS2021_cfi.py

clacaputo · 2022-06-28T12:20:26Z

-reconstruction

clacaputo · 2022-07-31T07:07:17Z

+reconstruction

bsunanda · 2022-07-31T09:37:22Z

+geometry

forthommel · 2022-08-01T08:32:10Z

@ahmad3213, @emanueleusai, @jfernan2, @pmandrik, @micsucmed, @rvenditti, @davidlange6, @perrotta, @rappoccio, @qliphy, @fabiocos, @civanch, @mdhildreth, any help needed in your (second) review?

emanueleusai · 2022-08-02T06:05:35Z

+1

forthommel · 2022-08-02T06:55:40Z

Thanks, @emanueleusai!
@davidlange6, @perrotta, @rappoccio, @qliphy, @fabiocos, @civanch, @mdhildreth, any help needed in your (second) review?

civanch · 2022-08-02T08:21:38Z

+1

forthommel · 2022-08-02T08:44:02Z

Thanks, @civanch!
@davidlange6, @perrotta, @rappoccio, @qliphy, @fabiocos, any help needed in your (second) review?

forthommel · 2022-08-02T21:38:13Z

I know you have a lot on your stack nowadays, but would it be possible to converge on this PR anytime soon? It has been almost two months since it was opened.

qliphy · 2022-08-04T01:08:16Z

+1

cmsbuild · 2022-08-04T01:08:39Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

forthommel · 2022-08-04T05:28:48Z

Many thanks, @qliphy !

cmsbuild added this to the CMSSW_12_5_X milestone Jun 7, 2022

cmsbuild added code-checks-pending dqm-pending geometry-pending new-package-pending operations-pending orp-pending pending-signatures reconstruction-pending simulation-pending tests-pending labels Jun 7, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 7, 2022

cmsbuild added tests-started and removed tests-pending labels Jun 8, 2022

cmsbuild added tests-rejected and removed tests-started labels Jun 8, 2022

cmsbuild mentioned this pull request Jun 8, 2022

Adding HZZ electron MVA ID #37429

Closed

civanch reviewed Jun 11, 2022

View reviewed changes

SimPPS/Configuration/python/directSimPPS_cff.py Outdated Show resolved Hide resolved

civanch reviewed Jun 11, 2022

View reviewed changes

SimPPS/DirectSimProducer/python/profiles_2021_cff.py Outdated Show resolved Hide resolved

civanch reviewed Jun 11, 2022

View reviewed changes

SimPPS/DirectSimProducer/python/simPPS2021_cfi.py Outdated Show resolved Hide resolved

cmsbuild mentioned this pull request Jun 27, 2022

Remove Phase-2 bricked pixels specific reconstruction code from CMSSW #38529

Merged

cmsbuild added reconstruction-rejected and removed reconstruction-pending labels Jun 28, 2022

cmsbuild mentioned this pull request Jun 30, 2022

Add new Run3_ckfPixelLessStep to StandardSequences #38563

Closed

cmsbuild added the daq-approved label Jul 30, 2022

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Jul 31, 2022

cmsbuild added geometry-approved and removed geometry-pending labels Jul 31, 2022

cmsbuild mentioned this pull request Jul 31, 2022

Migration of input collection parameters to untracked for CTPPS and DTMonitorModule #38854

Closed

cmsbuild added dqm-approved and removed dqm-pending labels Aug 2, 2022

cmsbuild added simulation-approved and removed simulation-pending labels Aug 2, 2022

cmsbuild added fully-signed operations-approved orp-approved and removed operations-pending pending-signatures orp-pending labels Aug 4, 2022

cmsbuild merged commit f10295e into cms-sw:master Aug 4, 2022

forthommel deleted the pps-direct_simu_stdseq-12_5_X branch August 4, 2022 05:28

cmsbuild mentioned this pull request Aug 4, 2022

Update the autoSkim.py for new HLT menu v1.3 #38933

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[PPS] Direct simulation as sequence #38280

[PPS] Direct simulation as sequence #38280

forthommel commented Jun 7, 2022 •

edited

cmsbuild commented Jun 7, 2022

cmsbuild commented Jun 7, 2022

jpata commented Jun 8, 2022

cmsbuild commented Jun 8, 2022

clacaputo commented Jun 28, 2022

clacaputo commented Jul 31, 2022

bsunanda commented Jul 31, 2022

forthommel commented Aug 1, 2022

emanueleusai commented Aug 2, 2022

forthommel commented Aug 2, 2022

civanch commented Aug 2, 2022

forthommel commented Aug 2, 2022

forthommel commented Aug 2, 2022

qliphy commented Aug 4, 2022

cmsbuild commented Aug 4, 2022

forthommel commented Aug 4, 2022

[PPS] Direct simulation as sequence #38280

[PPS] Direct simulation as sequence #38280

Conversation

forthommel commented Jun 7, 2022 • edited

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Jun 7, 2022

cmsbuild commented Jun 7, 2022

jpata commented Jun 8, 2022

cmsbuild commented Jun 8, 2022

RelVals

RelVals-INPUT

AddOn Tests

clacaputo commented Jun 28, 2022

clacaputo commented Jul 31, 2022

bsunanda commented Jul 31, 2022

forthommel commented Aug 1, 2022

emanueleusai commented Aug 2, 2022

forthommel commented Aug 2, 2022

civanch commented Aug 2, 2022

forthommel commented Aug 2, 2022

forthommel commented Aug 2, 2022

qliphy commented Aug 4, 2022

cmsbuild commented Aug 4, 2022

forthommel commented Aug 4, 2022

forthommel commented Jun 7, 2022 •

edited