PyReleaseValidation production-like workflow for Run3 (2021) #28773

fabiocos · 2020-01-21T17:31:43Z

PR description:

This PR proposes a special set of Run3 (2021) workflows in a "production-like" setup, mimicking what already available for Phase2 by @kpedro88 . The purpose is to provide a tool for performance evaluation ready out-of-the-box, and it is not added to the standard set of workflows to avoid inflating it further for such a specific use case (so --what upgrade option needs to be invoked).

PR validation:

The added configuration cannot be tested directly by the bot. Here is the output:

12:35 cmsdev25 653> runTheMatrix.py --what upgrade -l 11634.21 --dryRun
ignoring non-requested file relval_standard
ignoring non-requested file relval_highstats
ignoring non-requested file relval_pileup
ignoring non-requested file relval_generator
ignoring non-requested file relval_extendedgen
ignoring non-requested file relval_production
ignoring non-requested file relval_ged
processing relval_upgrade
ignoring non-requested file relval_2017
ignoring non-requested file relval_2026
ignoring non-requested file relval_identity
ignoring non-requested file relval_machine
ignoring non-requested file relval_unsch
ignoring non-requested file relval_premix
Running in 4 thread(s)

Preparing to run 11634.21 TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_ProdLike_2021+RecoFull_ProdLike_2021+MiniAODFullGlobal_ProdLike_2021

# in: /build/fabiocos/111X/prodlikeWF/CMSSW_11_1_X_2020-01-21-2300/work dryRun for 'cd 11634.21_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_ProdLike_2021+RecoFull_ProdLike_2021+MiniAODFullGlobal_ProdLike_2021
 cmsDriver.py TTbar_14TeV_TuneCUETP8M1_cfi  --conditions auto:phase1_2021_realistic -n 10 --era Run3 --eventcontent FEVTDEBUG --relval 9000,100 -s GEN,SIM --datatier GEN-SIM --beamspot Run3RoundOptics25ns13TeVLowSigmaZ --geometry DB:Extended --fileout file:step1.root  > step1_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_ProdLike_2021+RecoFull_ProdLike_2021+MiniAODFullGlobal_ProdLike_2021.log  2>&1
 

# in: /build/fabiocos/111X/prodlikeWF/CMSSW_11_1_X_2020-01-21-2300/work dryRun for 'cd 11634.21_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_ProdLike_2021+RecoFull_ProdLike_2021+MiniAODFullGlobal_ProdLike_2021
 cmsDriver.py step2  --conditions auto:phase1_2021_realistic -s DIGI,L1,DIGI2RAW,HLT:@relval2021 --datatier GEN-SIM-DIGI-RAW -n 10 --geometry DB:Extended --era Run3 --eventcontent RAWSIM --filein  file:step1.root  --fileout file:step2.root  > step2_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_ProdLike_2021+RecoFull_ProdLike_2021+MiniAODFullGlobal_ProdLike_2021.log  2>&1
 

# in: /build/fabiocos/111X/prodlikeWF/CMSSW_11_1_X_2020-01-21-2300/work dryRun for 'cd 11634.21_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_ProdLike_2021+RecoFull_ProdLike_2021+MiniAODFullGlobal_ProdLike_2021
 cmsDriver.py step3  --conditions auto:phase1_2021_realistic -n 10 --era Run3 --eventcontent AODSIM --runUnscheduled  -s RAW2DIGI,L1Reco,RECO,RECOSIM --datatier AODSIM --geometry DB:Extended --filein  file:step2.root  --fileout file:step3.root  > step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_ProdLike_2021+RecoFull_ProdLike_2021+MiniAODFullGlobal_ProdLike_2021.log  2>&1
 

# in: /build/fabiocos/111X/prodlikeWF/CMSSW_11_1_X_2020-01-21-2300/work dryRun for 'cd 11634.21_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_ProdLike_2021+RecoFull_ProdLike_2021+MiniAODFullGlobal_ProdLike_2021
 cmsDriver.py step4  --conditions auto:phase1_2021_realistic -n 10 --era Run3 --eventcontent MINIAODSIM --runUnscheduled  -s PAT --datatier MINIAODSIM --geometry DB:Extended --filein  file:step3.root  --fileout file:step4.root  > step4_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_ProdLike_2021+RecoFull_ProdLike_2021+MiniAODFullGlobal_ProdLike_2021.log  2>&1
 
11634.21_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_ProdLike_2021+RecoFull_ProdLike_2021+MiniAODFullGlobal_ProdLike_2021 Step0-PASSED Step1-PASSED Step2-PASSED Step3-PASSED  - time date Sun Jan 26 12:35:14 2020-date Sun Jan 26 12:35:14 2020; exit: 0 0 0 0
1 1 1 1 tests passed, 0 0 0 0 failed

cmsbuild · 2020-01-21T17:32:11Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-01-21T17:38:13Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28773/13427

This PR adds an extra 20KB to repository

cmsbuild · 2020-01-21T17:38:41Z

A new Pull Request was created by @fabiocos (Fabio Cossutti) for master.

It involves the following packages:

Configuration/PyReleaseValidation

@chayanit, @cmsbuild, @pgunnell, @kpedro88, @zhenhu can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald this is something you requested to watch as well.
@davidlange6, @silviodonato, @fabiocos you are the release manager for this.

cms-bot commands are listed here

fabiocos · 2020-01-21T17:40:18Z

please test

testing only that nothing old is broken, the addition needs to be tested with --what upgrade option for runTheMatrix.py

fabiocos · 2020-01-21T17:40:33Z

@davidlange6 @slava77 FYI

cmsbuild · 2020-01-21T17:40:41Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4300/console Started: 2020/01/21 18:43

cmsbuild · 2020-01-21T17:45:32Z

-1

Tested at: c67d7be

CMSSW: CMSSW_11_1_X_2020-01-21-1100
SCRAM_ARCH: slc7_amd64_gcc820
You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13ae09/4300/summary.html

I found follow errors while testing this PR

Failed tests: Python3

Python3:

I found errors:

*** Error compiling 'src/Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py'...

cmsbuild · 2020-01-21T17:45:34Z

Comparison not run due to Build errors/Fireworks only changes/No short matrix requested (RelVals and Igprof tests were also skipped)

Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py

kpedro88 · 2020-01-21T19:55:32Z

@fabiocos I think the test failure is because of incorrect indentation, but see my review comment #28773 (comment) for a way to simplify the PR

cmsbuild · 2020-01-22T17:09:28Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4316/console Started: 2020/01/22 18:19

cmsbuild · 2020-01-22T18:27:24Z

+1
Tested at: f2cb77a
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13ae09/4316/summary.html
CMSSW: CMSSW_11_1_X_2020-01-22-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-01-22T18:27:28Z

Comparison job queued.

cmsbuild · 2020-01-22T20:00:56Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13ae09/4316/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2697090
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2696743
DQMHistoTests: Total skipped: 346
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

fabiocos · 2020-01-24T15:42:18Z

@kpedro88 @chayanit further comments?

chayanit · 2020-01-24T15:48:46Z

@fabiocos , I have no comments about the code. For my knowledge, what difference is between prod-like and baseline workflows in general?

fabiocos · 2020-01-24T15:52:44Z

@chayanit the main difference is the absence of DQM, and the output tiers are those used in production. For performance evaluation this may make a difference

chayanit · 2020-01-24T15:54:38Z

+1

@fabiocos thanks for your prompt response.

kpedro88 · 2020-01-24T18:17:36Z

@fabiocos can you update the PR description to indicate the new workflow is now 11634.21 rather than 11634.22?

kpedro88 · 2020-01-24T18:17:41Z

+upgrade

cmsbuild · 2020-01-24T18:18:02Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @silviodonato, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

fabiocos · 2020-01-26T11:36:40Z

@kpedro88 I have updated the description to leave a clean reference

silviodonato · 2020-01-27T17:51:39Z

+1

cmsbuild added this to the CMSSW_11_1_X milestone Jan 21, 2020

cmsbuild added code-checks-pending comparison-pending orp-pending pdmv-pending pending-signatures tests-pending upgrade-pending labels Jan 21, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 21, 2020

cmsbuild added tests-started and removed tests-pending labels Jan 21, 2020

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels Jan 21, 2020

kpedro88 reviewed Jan 21, 2020

View reviewed changes

Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py Outdated Show resolved Hide resolved

cmsbuild mentioned this pull request Jan 21, 2020

add TestOldDigi workflow #28775

Merged

fabiocos added 2 commits January 22, 2020 09:57

Add ProdLike workflow for Run3

a7cb070

remove AlcaReco from prod-like workflow

49ad23e

fabiocos force-pushed the fc-prodlikeWF branch from c67d7be to 49ad23e Compare January 22, 2020 09:34

cmsbuild added tests-started and removed tests-pending labels Jan 22, 2020

cmsbuild added tests-approved and removed tests-started labels Jan 22, 2020

cmsbuild added comparison-available and removed comparison-pending labels Jan 22, 2020

cmsbuild added pdmv-approved and removed pdmv-pending labels Jan 24, 2020

cmsbuild removed the pending-signatures label Jan 24, 2020

cmsbuild added fully-signed upgrade-approved and removed upgrade-pending labels Jan 24, 2020

cmsbuild added orp-approved and removed orp-pending labels Jan 27, 2020

cmsbuild merged commit ceae341 into cms-sw:master Jan 27, 2020

fabiocos deleted the fc-prodlikeWF branch January 28, 2020 10:57

silviodonato mentioned this pull request Mar 2, 2020

GEM validation package update #29042

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PyReleaseValidation production-like workflow for Run3 (2021) #28773

PyReleaseValidation production-like workflow for Run3 (2021) #28773

fabiocos commented Jan 21, 2020 •

edited

cmsbuild commented Jan 21, 2020

cmsbuild commented Jan 21, 2020

cmsbuild commented Jan 21, 2020

fabiocos commented Jan 21, 2020

fabiocos commented Jan 21, 2020

cmsbuild commented Jan 21, 2020 •

edited

cmsbuild commented Jan 21, 2020

cmsbuild commented Jan 21, 2020

kpedro88 commented Jan 21, 2020

cmsbuild commented Jan 22, 2020 •

edited

cmsbuild commented Jan 22, 2020

cmsbuild commented Jan 22, 2020

cmsbuild commented Jan 22, 2020

fabiocos commented Jan 24, 2020

chayanit commented Jan 24, 2020

fabiocos commented Jan 24, 2020

chayanit commented Jan 24, 2020

kpedro88 commented Jan 24, 2020

kpedro88 commented Jan 24, 2020

cmsbuild commented Jan 24, 2020

fabiocos commented Jan 26, 2020

silviodonato commented Jan 27, 2020

PyReleaseValidation production-like workflow for Run3 (2021) #28773

PyReleaseValidation production-like workflow for Run3 (2021) #28773

Conversation

fabiocos commented Jan 21, 2020 • edited

PR description:

PR validation:

cmsbuild commented Jan 21, 2020

cmsbuild commented Jan 21, 2020

cmsbuild commented Jan 21, 2020

fabiocos commented Jan 21, 2020

fabiocos commented Jan 21, 2020

cmsbuild commented Jan 21, 2020 • edited

cmsbuild commented Jan 21, 2020

cmsbuild commented Jan 21, 2020

kpedro88 commented Jan 21, 2020

cmsbuild commented Jan 22, 2020 • edited

cmsbuild commented Jan 22, 2020

cmsbuild commented Jan 22, 2020

cmsbuild commented Jan 22, 2020

fabiocos commented Jan 24, 2020

chayanit commented Jan 24, 2020

fabiocos commented Jan 24, 2020

chayanit commented Jan 24, 2020

kpedro88 commented Jan 24, 2020

kpedro88 commented Jan 24, 2020

cmsbuild commented Jan 24, 2020

fabiocos commented Jan 26, 2020

silviodonato commented Jan 27, 2020

fabiocos commented Jan 21, 2020 •

edited

cmsbuild commented Jan 21, 2020 •

edited

cmsbuild commented Jan 22, 2020 •

edited