Common pp_on_AA modifier for 2018 and run 3 AA eras #31935

mandrenguyen · 2020-10-25T16:37:00Z

Use a common modifier 'pp_on_AA' for both Run2_2018_pp_on_AA and Run3_pp_on_PbPb eras.
Run3_pp_on_PbPb was initially implemented as a superset of Run2_2018_pp_on_AA, but they were factorized in #31668
This PR restores the previous behavior in a cleaner way.
Run2_2018_pp_on_AA workflows should be unaffected, while Run3_pp_on_PbPb picks up many customizations that were dropped.

The lighter systems (pPb, XeXe, pp ref and peripheral PbPb) are not included in the common modifier.
One could imagine further adding an overarching process modifier for all heavy ion eras, since they have some common features.
This is left for a possible future PR.

I tested with wfs 158 and 159.

cmsbuild · 2020-10-25T16:37:32Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-10-25T16:44:17Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31935/19376

This PR adds an extra 476KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DQMOffline/Configuration/python/DQMOffline_cff.py modified in PR(s): Patatrack integration - Pixel vertex reconstruction (11/N) #31723, Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File PhysicsTools/PatAlgos/python/slimming/MicroEventContent_cff.py modified in PR(s): Adding vertex recovery step for 2018 heavy ion miniAOD production. #31676, Add low-pT electrons to MINIAOD, update ID, improve end user experience #31220, HeavyIon Reaction Plane MiniAOD #31129
- File PhysicsTools/PatAlgos/python/slimming/applySubstructure_cff.py modified in PR(s): Speed up HI miniAOD by removing unneeded features #31893
- File PhysicsTools/PatAlgos/python/slimming/isolatedTracks_cfi.py modified in PR(s): Add track-based isolation, recHit energy sum to IsolatedTrack class #31399
- File PhysicsTools/PatAlgos/python/slimming/slimming_cff.py modified in PR(s): Adding vertex recovery step for 2018 heavy ion miniAOD production. #31676, Add low-pT electrons to MINIAOD, update ID, improve end user experience #31220, HeavyIon Reaction Plane MiniAOD #31129
- File RecoBTag/ImpactParameter/python/impactParameterTagInfos_cfi.py modified in PR(s): Speed up HI miniAOD by removing unneeded features #31893
- File RecoEgamma/Configuration/python/RecoEgamma_EventContent_cff.py modified in PR(s): Add low-pT electrons to MINIAOD, update ID, improve end user experience #31220
- File RecoTauTag/Configuration/python/boostedHPSPFTaus_cff.py modified in PR(s): Speed up HI miniAOD by removing unneeded features #31893
- File RecoTracker/IterativeTracking/python/DetachedQuadStep_cff.py modified in PR(s): Track classifier using TensorFlow #31682
- File RecoTracker/IterativeTracking/python/DetachedTripletStep_cff.py modified in PR(s): Track classifier using TensorFlow #31682
- File RecoTracker/IterativeTracking/python/HighPtTripletStep_cff.py modified in PR(s): Track classifier using TensorFlow #31682
- File RecoTracker/IterativeTracking/python/InitialStep_cff.py modified in PR(s): Track classifier using TensorFlow #31682
- File RecoTracker/IterativeTracking/python/JetCoreRegionalStep_cff.py modified in PR(s): Track classifier using TensorFlow #31682
- File RecoTracker/IterativeTracking/python/LowPtQuadStep_cff.py modified in PR(s): Track classifier using TensorFlow #31682
- File RecoTracker/IterativeTracking/python/LowPtTripletStep_cff.py modified in PR(s): Track classifier using TensorFlow #31682
- File RecoTracker/IterativeTracking/python/MixedTripletStep_cff.py modified in PR(s): Track classifier using TensorFlow #31682
- File RecoTracker/IterativeTracking/python/PixelLessStep_cff.py modified in PR(s): Track classifier using TensorFlow #31682
- File RecoTracker/IterativeTracking/python/PixelPairStep_cff.py modified in PR(s): Track classifier using TensorFlow #31682
- File RecoTracker/IterativeTracking/python/TobTecStep_cff.py modified in PR(s): Track classifier using TensorFlow #31682
- File RecoVertex/PrimaryVertexProducer/python/OfflinePrimaryVertices_cfi.py modified in PR(s): Adding vertex recovery step for 2018 heavy ion miniAOD production. #31676
- File SimCalorimetry/HcalTrigPrimProducers/python/hcaltpdigi_cff.py modified in PR(s): HCAL Pulse Filtering for OOT PU Subtraction #31661
- File Validation/Configuration/python/globalValidation_cff.py modified in PR(s): Patatrack integration - Pixel local reconstruction (9/N) #31721

cmsbuild · 2020-10-25T16:44:52Z

A new Pull Request was created by @mandrenguyen (Matthew Nguyen) for master.

It involves the following packages:

Alignment/CommonAlignmentProducer
CalibMuon/DTCalibration
CalibTracker/SiPixelQuality
Calibration/HcalAlCaRecoProducers
Calibration/TkAlCaRecoProducers
Configuration/Eras
Configuration/EventContent
Configuration/ProcessModifiers
Configuration/StandardSequences
DQM/Physics
DQM/SiStripMonitorClient
DQM/TrackingMonitorClient
DQM/TrackingMonitorSource
DQMOffline/Configuration
DQMOffline/EGamma
DQMOffline/JetMET
DQMOffline/RecoB
DQMOffline/Trigger
HLTriggerOffline/B2G
HLTriggerOffline/Common
PhysicsTools/PatAlgos
RecoBTag/ImpactParameter
RecoBTag/SecondaryVertex
RecoBTag/SoftLepton
RecoEcal/Configuration
RecoEcal/EgammaClusterProducers
RecoEgamma/Configuration
RecoEgamma/EgammaElectronProducers
RecoEgamma/EgammaIsolationAlgos
RecoEgamma/EgammaPhotonProducers
RecoEgamma/EgammaTools
RecoHI/HiCentralityAlgos
RecoJets/Configuration
RecoJets/JetAssociationProducers
RecoJets/JetProducers
RecoLocalCalo/Configuration
RecoLocalCalo/HcalRecProducers
RecoMET/Configuration
RecoMuon/MuonIdentification
RecoParticleFlow/Configuration
RecoParticleFlow/PFProducer
RecoParticleFlow/PFTracking
RecoPixelVertexing/PixelTriplets
RecoTauTag/Configuration
RecoTracker/Configuration
RecoTracker/ConversionSeedGenerators
RecoTracker/IterativeTracking
RecoTracker/TkSeedGenerator
RecoTracker/TkTrackingRegions
RecoVertex/AdaptiveVertexFinder
RecoVertex/PrimaryVertexProducer
RecoVertex/V0Producer
SimCalorimetry/HcalTrigPrimProducers
SimGeneral/Configuration
TrackingTools/GsfTracking
Validation/Configuration
Validation/EcalClusters

@andrius-k, @rekovic, @fioriNTU, @tlampen, @pohsun, @santocch, @perrotta, @civanch, @yuanchao, @silviodonato, @ErnestaP, @cmsbuild, @davidlange6, @tocheng, @jfernan2, @mdhildreth, @slava77, @jpata, @qliphy, @fabiocos, @benkrikler, @kmaeshima, @christopheralanwest, @franzoni can you please review it and eventually sign? Thanks.
@emilbols, @gouskos, @makortel, @yslai, @felicepantaleo, @jainshilpi, @jandrea, @rappoccio, @apsallid, @argiro, @Martin-Grunewald, @Fedespring, @thomreis, @tlampen, @lgray, @abbiendi, @ebrondol, @seemasharmafnal, @venturia, @pakhotin, @mmarionncern, @slomeo, @smoortga, @hdelanno, @JyothsnaKomaragiri, @ahinzmann, @abdoulline, @JanFSchulte, @jhgoh, @dgulhan, @missirol, @yenjie, @cericeci, @ferencek, @rociovilar, @dkotlins, @trocino, @cbernet, @fioriNTU, @Sam-Harper, @HuguesBrun, @sroychow, @afiqaize, @GiacomoSguazzoni, @tocheng, @VinInn, @jdamgov, @bellan, @nhanvtran, @gkasieczka, @rovere, @threus, @schoef, @mschrode, @idebruyn, @mmusich, @mtosi, @fabiocos, @clelange, @arossi83, @jdolen, @adewit, @rchatter, @riga, @battibass, @varuns23, @jazzitup, @calderona, @hatakeyamak, @sobhatta, @lecriste, @kurtejung, @gpetruc, @mariadalfonso, @yetkinyilmaz, @andrzejnovak, @folguera, @tvami this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

jfernan2 · 2020-10-26T09:13:05Z

please test

cmsbuild · 2020-10-26T09:13:35Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-10-25-2300/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10280/console Started: 2020/10/26 10:15

cmsbuild · 2020-10-26T12:06:28Z

+1
Tested at: 8a02bf5
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e519da/10280/summary.html
CMSSW: CMSSW_11_2_X_2020-10-25-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-10-26T12:06:31Z

Comparison job queued.

cmsbuild · 2020-10-26T13:22:22Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e519da/10280/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2544110
DQMHistoTests: Total failures: 13
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2544075
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 149 log files, 22 edm output root files, 35 DQM output files

jfernan2 · 2020-10-26T14:12:08Z

I see Static Check build errors unrelated to this PR, does anybody have any idea of their source?

jpata · 2020-11-02T11:18:19Z

+reconstruction

adds a modifier pp_on_AA instead of pp_on_AA_2018 for HI workflows
standard reco is not affected
HI test workflows 158.0 and 159.0 run successfully
static analyzer failures are unrelated to the changes

silviodonato · 2020-11-02T14:34:45Z

+operations

civanch · 2020-11-02T22:11:01Z

+1

christopheralanwest · 2020-11-03T03:12:00Z

+alca

rekovic · 2020-11-03T10:16:16Z

+1

slava77 · 2020-11-03T17:20:49Z

@qliphy @silviodonato
may I suggest not to wait for analysis signature here.
Please consider.
Thank you.

qliphy · 2020-11-04T00:26:23Z

+1

santocch · 2020-11-04T10:32:01Z

+1

cmsbuild · 2020-11-04T10:32:29Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

cmsbuild added this to the CMSSW_11_2_X milestone Oct 25, 2020

cmsbuild added alca-pending analysis-pending code-checks-pending comparison-pending dqm-pending l1-pending operations-pending orp-pending pending-signatures reconstruction-pending simulation-pending tests-pending labels Oct 25, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 25, 2020

cmsbuild added tests-started and removed tests-pending labels Oct 26, 2020

cmsbuild added tests-approved and removed tests-started labels Oct 26, 2020

cmsbuild added comparison-available and removed comparison-pending labels Oct 26, 2020

cmsbuild mentioned this pull request Nov 2, 2020

PuppiMET with PUPPIv15 and UL17 PUID #31927

Merged

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Nov 2, 2020

cmsbuild added operations-approved and removed operations-pending labels Nov 2, 2020

cmsbuild added simulation-approved and removed simulation-pending labels Nov 2, 2020

cmsbuild added alca-approved and removed alca-pending labels Nov 3, 2020

cmsbuild mentioned this pull request Nov 3, 2020

Make SiPixelStatusProducer a stream EDProducer for concurrent lumiblock processing #32012

Merged

cmsbuild added l1-approved and removed l1-pending labels Nov 3, 2020

qliphy merged commit caa4836 into cms-sw:master Nov 4, 2020

cmsbuild added orp-approved and removed orp-pending labels Nov 4, 2020

cmsbuild added analysis-approved fully-signed and removed analysis-pending pending-signatures labels Nov 4, 2020

slava77 mentioned this pull request Nov 12, 2020

Add low-pT electrons to MINIAOD, update ID, improve end user experience #31220

Merged

slava77 mentioned this pull request Dec 18, 2020

switch modifiers to pp_on_AA for hiEvtPlane #32541

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Common pp_on_AA modifier for 2018 and run 3 AA eras #31935

Common pp_on_AA modifier for 2018 and run 3 AA eras #31935

mandrenguyen commented Oct 25, 2020

cmsbuild commented Oct 25, 2020

cmsbuild commented Oct 25, 2020

cmsbuild commented Oct 25, 2020

jfernan2 commented Oct 26, 2020

cmsbuild commented Oct 26, 2020 •

edited

Loading

cmsbuild commented Oct 26, 2020

cmsbuild commented Oct 26, 2020

cmsbuild commented Oct 26, 2020

jfernan2 commented Oct 26, 2020

jpata commented Nov 2, 2020

silviodonato commented Nov 2, 2020

civanch commented Nov 2, 2020

christopheralanwest commented Nov 3, 2020

rekovic commented Nov 3, 2020

slava77 commented Nov 3, 2020

qliphy commented Nov 4, 2020

santocch commented Nov 4, 2020

cmsbuild commented Nov 4, 2020

Common pp_on_AA modifier for 2018 and run 3 AA eras #31935

Common pp_on_AA modifier for 2018 and run 3 AA eras #31935

Conversation

mandrenguyen commented Oct 25, 2020

cmsbuild commented Oct 25, 2020

cmsbuild commented Oct 25, 2020

cmsbuild commented Oct 25, 2020

jfernan2 commented Oct 26, 2020

cmsbuild commented Oct 26, 2020 • edited Loading

cmsbuild commented Oct 26, 2020

cmsbuild commented Oct 26, 2020

cmsbuild commented Oct 26, 2020

jfernan2 commented Oct 26, 2020

jpata commented Nov 2, 2020

silviodonato commented Nov 2, 2020

civanch commented Nov 2, 2020

christopheralanwest commented Nov 3, 2020

rekovic commented Nov 3, 2020

slava77 commented Nov 3, 2020

qliphy commented Nov 4, 2020

santocch commented Nov 4, 2020

cmsbuild commented Nov 4, 2020

cmsbuild commented Oct 26, 2020 •

edited

Loading