Add Run3 ALCA Express/Prompt workflow #38681

tocheng · 2022-07-11T08:58:23Z

PR description:

Add one Express-like and one Prompt-like ALCA workflows covering ALCRECO, ALCAPROMPT to Harvestor using Run-3 data.

2022A SingleMu data is used for Express like workflow(2022B SingleMu data seems have problem accessing through xrootd)

2022B ZeroBias data is used for Prompt like (MinimumBias data has zero events in 2022B Collision runs so far, so use ZeroBias instead)

PR validation:

The two workflows are tested by runTheMatrix

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Not a backport

tvami · 2022-07-11T09:00:20Z

thanks @tocheng I think it would be useful to backport this to 12_4_X too, the pp data taking is happening with that, so the wf-s being tested here would make sense there

tvami · 2022-07-11T09:00:25Z

assign alca

tvami · 2022-07-11T09:00:33Z

type new-feature

tvami · 2022-07-11T09:01:32Z

test parameters:

workflows = 1001.3,1002.3

tvami · 2022-07-11T09:01:55Z

@cmsbuild , please test

cmsbuild · 2022-07-11T09:03:23Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38681/30969

This PR adds an extra 52KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Reorganize FastSim Run3 workflow #38660

cmsbuild · 2022-07-11T09:03:45Z

New categories assigned: alca

@yuanchao,@francescobrivio,@malbouis,@tvami,@ChrisMisan you have been requested to review this Pull request/Issue and eventually sign? Thanks

cmsbuild · 2022-07-11T09:03:48Z

A new Pull Request was created by @tocheng (Tongguang) for master.

It involves the following packages:

Configuration/PyReleaseValidation (pdmv, upgrade)

@malbouis, @yuanchao, @jordan-martins, @bbilin, @AdrianoDee, @srimanob, @kskovpen, @tvami, @ChrisMisan, @francescobrivio can you please review it and eventually sign? Thanks.
@makortel, @kpedro88, @fabiocos, @Martin-Grunewald, @missirol, @trtomei, @beaucero, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

mmusich · 2022-07-11T09:58:17Z

Configuration/PyReleaseValidation/python/relval_production.py

@@ -12,6 +12,8 @@
 workflows[1000] = [ '',['RunMinBias2011A','TIER0','SKIMD','HARVESTDfst2','ALCASPLIT']]
 workflows[1001] = [ '',['RunMinBias2011A','TIER0EXP','ALCAEXP','ALCAHARVDSIPIXELCALRUN1','ALCAHARVD1','ALCAHARVD2','ALCAHARVD3','ALCAHARVD4','ALCAHARVD5','ALCAHARVD7','ALCAHARVD8']]
 workflows[1001.2] = [ '',['RunZeroBias2017F','TIER0EXPRUN2','ALCAEXPRUN2','ALCAHARVDSIPIXELCAL','ALCAHARVDSIPIXELCALLA','ALCAHARVD4','ALCAHARVDSIPIXELALIHG']]
+workflows[1001.3] = [ '',['RunSingleMuon2022A','TIER0EXPRUN3','ALCAEXPRUN3','ALCAHARVDEXPRUN3']]
+workflows[1002.3] = [ '',['RunZeroBias2022B','TIER0PROMPTRUN3','ALCAEXPRUN3','ALCAHARVDEXPRUN3']]


why is the "Prompt" configuration re-running the ALCAEXPRUN3 block? Shouldn't it rather run the run-3 equivalent of ALCASPLIT ?

cmsbuild · 2022-07-11T13:07:09Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-228118/26124/summary.html
COMMIT: 1ec5b2a
CMSSW: CMSSW_12_5_X_2022-07-10-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38681/26124/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

136.858136.858_RunNoBPTX2018A+RunNoBPTX2018A+HLTDR2_2018+RECODR2_2018reHLTAlCaTkCosmics_Offline+HARVEST2018/step2_RunNoBPTX2018A+RunNoBPTX2018A+HLTDR2_2018+RECODR2_2018reHLTAlCaTkCosmics_Offline+HARVEST2018.log

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-228118/1001.3_RunSingleMuon2022A+RunSingleMuon2022A+TIER0EXPRUN3+ALCAEXPRUN3+ALCAHARVDEXPRUN3
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-228118/1002.3_RunZeroBias2022B+RunZeroBias2022B+TIER0PROMPTRUN3+ALCAEXPRUN3+ALCAHARVDEXPRUN3

Summary:

No significant changes to the logs found
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3655970
DQMHistoTests: Total failures: 9
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3655939
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.43 KiB( 49 files compared)
DQMHistoSizes: changed ( 138.5 ): 0.215 KiB MessageLogger/Errors
DQMHistoSizes: changed ( 138.5 ): 0.215 KiB MessageLogger/Warnings
Checked 208 log files, 45 edm output root files, 50 DQM output files
TriggerResults: no differences found

ChrisMisan · 2022-07-11T13:59:34Z

file opening error in 136.858

ChrisMisan · 2022-07-11T14:03:26Z

@cmsbuild, please test

cmsbuild · 2022-07-20T03:54:41Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38681/31146

This PR adds an extra 52KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Update BeamSpot in Run3 MC GTs + swap Run3RoundOptics25ns13TeVLowSigmaZ with Realistic25ns13p6TeVEarly2022Collision vtx smearing #38760, Ari changes #38758, Reorganize FastSim Run3 workflow #38660, Don't recluster jets in HI miniAOD for Run 3 #38711, Add Run3 wfs (NanoV10) #38733, Add CPPF DQM modules #38564

cmsbuild · 2022-07-20T03:55:06Z

Pull request #38681 was updated. @malbouis, @yuanchao, @jordan-martins, @bbilin, @cmsbuild, @AdrianoDee, @srimanob, @kskovpen, @tvami, @ChrisMisan, @francescobrivio can you please check and sign again.

tvami · 2022-07-20T07:11:16Z

@cmsbuild , please test

cmsbuild · 2022-07-20T11:05:20Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-228118/26333/summary.html
COMMIT: 96482c8
CMSSW: CMSSW_12_5_X_2022-07-19-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38681/26333/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-228118/1001.3_RunSingleMuon2022B+RunSingleMuon2022B+TIER0EXPRUN3+ALCAEXPRUN3+ALCAHARVDEXPRUN3
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-228118/1002.3_RunZeroBias2022B+RunZeroBias2022B+TIER0PROMPTRUN3+ALCASPLITRUN3+ALCAHARVDEXPRUN3

Summary:

No significant changes to the logs found
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3662417
DQMHistoTests: Total failures: 9
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3662386
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.398 KiB( 49 files compared)
DQMHistoSizes: changed ( 138.5 ): 0.199 KiB MessageLogger/Errors
DQMHistoSizes: changed ( 138.5 ): 0.199 KiB MessageLogger/Warnings
Checked 208 log files, 45 edm output root files, 50 DQM output files
TriggerResults: no differences found

tvami · 2022-07-20T13:08:24Z

+alca

tvami · 2022-07-20T13:08:37Z

@srimanob @kskovpen please resign, thanks!

srimanob · 2022-07-20T15:01:20Z

+Upgrade

kskovpen · 2022-07-20T20:42:55Z

+pdmv

cmsbuild · 2022-07-20T20:43:17Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-07-21T06:50:09Z

+1

cmsbuild added this to the CMSSW_12_5_X milestone Jul 11, 2022

cmsbuild added code-checks-pending orp-pending pdmv-pending pending-signatures tests-pending upgrade-pending labels Jul 11, 2022

cmsbuild added alca-pending new-feature labels Jul 11, 2022

cmsbuild added tests-started and removed tests-pending labels Jul 11, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 11, 2022

mmusich reviewed Jul 11, 2022

View reviewed changes

francescobrivio mentioned this pull request Jul 11, 2022

Profiling at T0 AlCa and DQM workflows #38198

Open

cmsbuild added tests-rejected and removed tests-started labels Jul 11, 2022

cmsbuild added tests-started and removed tests-rejected labels Jul 11, 2022

cmsbuild added code-checks-pending and removed code-checks-approved labels Jul 20, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 20, 2022

cmsbuild added tests-started and removed tests-pending labels Jul 20, 2022

cmsbuild added tests-approved and removed tests-started labels Jul 20, 2022

cmsbuild added alca-approved and removed alca-pending labels Jul 20, 2022

cmsbuild added upgrade-approved and removed upgrade-pending labels Jul 20, 2022

cmsbuild added fully-signed pdmv-approved and removed pending-signatures pdmv-pending labels Jul 20, 2022

cmsbuild added orp-approved and removed orp-pending labels Jul 21, 2022

cmsbuild merged commit 03acd7f into cms-sw:master Jul 21, 2022

tocheng mentioned this pull request Jul 21, 2022

Add Run3 ALCA Express/Prompt workflow for CMSSW 12_4_X #38819

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Run3 ALCA Express/Prompt workflow #38681

Add Run3 ALCA Express/Prompt workflow #38681

tocheng commented Jul 11, 2022

tvami commented Jul 11, 2022

tvami commented Jul 11, 2022

tvami commented Jul 11, 2022

tvami commented Jul 11, 2022

tvami commented Jul 11, 2022

cmsbuild commented Jul 11, 2022

cmsbuild commented Jul 11, 2022

cmsbuild commented Jul 11, 2022

mmusich Jul 11, 2022

cmsbuild commented Jul 11, 2022

ChrisMisan commented Jul 11, 2022 •

edited

ChrisMisan commented Jul 11, 2022

cmsbuild commented Jul 20, 2022

cmsbuild commented Jul 20, 2022

tvami commented Jul 20, 2022

cmsbuild commented Jul 20, 2022

tvami commented Jul 20, 2022

tvami commented Jul 20, 2022

srimanob commented Jul 20, 2022

kskovpen commented Jul 20, 2022

cmsbuild commented Jul 20, 2022

perrotta commented Jul 21, 2022

Add Run3 ALCA Express/Prompt workflow #38681

Add Run3 ALCA Express/Prompt workflow #38681

Conversation

tocheng commented Jul 11, 2022

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

tvami commented Jul 11, 2022

tvami commented Jul 11, 2022

tvami commented Jul 11, 2022

tvami commented Jul 11, 2022

tvami commented Jul 11, 2022

cmsbuild commented Jul 11, 2022

cmsbuild commented Jul 11, 2022

cmsbuild commented Jul 11, 2022

mmusich Jul 11, 2022

Choose a reason for hiding this comment

cmsbuild commented Jul 11, 2022

RelVals-INPUT

Comparison Summary

ChrisMisan commented Jul 11, 2022 • edited

ChrisMisan commented Jul 11, 2022

cmsbuild commented Jul 20, 2022

cmsbuild commented Jul 20, 2022

tvami commented Jul 20, 2022

cmsbuild commented Jul 20, 2022

Comparison Summary

tvami commented Jul 20, 2022

tvami commented Jul 20, 2022

srimanob commented Jul 20, 2022

kskovpen commented Jul 20, 2022

cmsbuild commented Jul 20, 2022

perrotta commented Jul 21, 2022

ChrisMisan commented Jul 11, 2022 •

edited