[12_4_X] Remove hardcoded trigger bits from Lumi ALCARECOs #39752

purbita-rahman · 2022-10-18T05:16:27Z

PR description:

Remove hardcoded trigger bits from AlCaRecos and include it as a key in the alcareco trigger bits tag in "ALCARECOAlCaPCCZeroBias" and "ALCARECOAlCaPCCRandom"

Two GTs were created to be included in this PR, with the correct AlCaRecoTriggerBits tag.
1- 124X_dataRun3_Express_frozen_v6 :
GlobalTag for Run3 data relvals (express GT) - identical to 124X_dataRun3_Express_v6 but with snapshot at 2022-10-04 14:22:26 (UTC)
Here is the link of difference of GT, old vs new : https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_dataRun3_Express_frozen_v4/124X_dataRun3_Express_frozen_v6

2- 124X_dataRun3_Prompt_frozen_v5 :
GlobalTag for Run3 data relvals (prompt GT) - identical to 124X_dataRun3_Prompt_v5 but with snapshot at 2022-10-04 14:19:51 (UTC)
Here is the link of difference of GT, old vs new : https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_dataRun3_Prompt_frozen_v4/124X_dataRun3_Prompt_frozen_v

backport of #39519

cmsbuild · 2022-10-18T05:16:47Z

A new Pull Request was created by @purbita-rahman (Purbita Rahman ) for CMSSW_12_4_X.

It involves the following packages:

Calibration/LumiAlCaRecoProducers (alca)
Configuration/AlCa (alca)

@malbouis, @yuanchao, @cmsbuild, @saumyaphor4252, @francescobrivio, @ChrisMisan, @tvami can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @mmusich, @fabiocos, @tocheng this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

Backported from Remove hardcoded trigger bits from Lumi ALCARECOs #39519

ChrisMisan · 2022-10-18T08:22:39Z

@purbita-rahman please see #39751 (comment)
Also, please follow the PR template.

ChrisMisan · 2022-10-18T15:45:45Z

please test

ChrisMisan · 2022-10-18T15:49:17Z

backport of #39519

ChrisMisan · 2022-10-18T15:50:47Z

please abort

ChrisMisan · 2022-10-18T15:50:57Z

test parameters:

workflows = 1020,1002.3,1002.4,1001.3

ChrisMisan · 2022-10-18T15:51:39Z

please test

cmsbuild · 2022-10-18T20:27:27Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ed6ed7/28340/summary.html
COMMIT: 0ff3d66
CMSSW: CMSSW_12_4_X_2022-10-18-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39752/28340/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test TestDQMOnlineClient-ecal_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-dt_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-beampixel_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-hcal_dqm_sourceclient had ERRORS
and more ...

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ed6ed7/1001.3_RunSingleMuon2022B+RunSingleMuon2022B+TIER0EXPRUN3+ALCAEXPRUN3+ALCAHARVDEXPRUN3
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ed6ed7/1002.3_RunZeroBias2022B+RunZeroBias2022B+TIER0PROMPTRUN3+ALCASPLITRUN3+ALCAHARVDEXPRUN3
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ed6ed7/1002.4_RunDoubleMuon2022B+RunDoubleMuon2022B+TIER0PROMPTRUN3+HARVESTPROMPTRUN3
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ed6ed7/1020.0_AlCaLumiPixels2021+AlCaLumiPixels2021+TIER0EXPLP+ALCAEXPLP+ALCAHARVLP+TIER0PROMPTLP

Summary:

No significant changes to the logs found
Reco comparison results: 15 differences found in the comparisons
DQMHistoTests: Total files compared: 54
DQMHistoTests: Total histograms compared: 3806485
DQMHistoTests: Total failures: 19867
DQMHistoTests: Total nulls: 7
DQMHistoTests: Total successes: 3786589
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.091 KiB( 53 files compared)
DQMHistoSizes: changed ( 138.5,... ): -0.008 KiB MessageLogger/Errors
DQMHistoSizes: changed ( 138.5,... ): -0.008 KiB MessageLogger/Warnings
DQMHistoSizes: changed ( 1002.4 ): 0.064 KiB SiStrip/MechanicalView
DQMHistoSizes: changed ( 1002.4 ): 0.059 KiB JetMET/SUSYDQM
Checked 224 log files, 45 edm output root files, 54 DQM output files
TriggerResults: found differences in 2 / 53 workflows

tvami · 2022-10-18T20:29:04Z

@cmsbuild , please test

failures are unrelated

cmsbuild · 2022-10-19T01:11:03Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ed6ed7/28347/summary.html
COMMIT: 0ff3d66
CMSSW: CMSSW_12_4_X_2022-10-18-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39752/28347/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test TestDQMOnlineClient-dt_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-beampixel_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-es_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-ecal_dqm_sourceclient had ERRORS
and more ...

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ed6ed7/1001.3_RunSingleMuon2022B+RunSingleMuon2022B+TIER0EXPRUN3+ALCAEXPRUN3+ALCAHARVDEXPRUN3
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ed6ed7/1002.3_RunZeroBias2022B+RunZeroBias2022B+TIER0PROMPTRUN3+ALCASPLITRUN3+ALCAHARVDEXPRUN3
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ed6ed7/1002.4_RunDoubleMuon2022B+RunDoubleMuon2022B+TIER0PROMPTRUN3+HARVESTPROMPTRUN3
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ed6ed7/1020.0_AlCaLumiPixels2021+AlCaLumiPixels2021+TIER0EXPLP+ALCAEXPLP+ALCAHARVLP+TIER0PROMPTLP

Summary:

No significant changes to the logs found
Reco comparison results: 15 differences found in the comparisons
DQMHistoTests: Total files compared: 54
DQMHistoTests: Total histograms compared: 3806485
DQMHistoTests: Total failures: 19866
DQMHistoTests: Total nulls: 8
DQMHistoTests: Total successes: 3786589
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.087 KiB( 53 files compared)
DQMHistoSizes: changed ( 138.5,... ): -0.008 KiB MessageLogger/Errors
DQMHistoSizes: changed ( 138.5,... ): -0.008 KiB MessageLogger/Warnings
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
DQMHistoSizes: changed ( 1002.4 ): 0.064 KiB SiStrip/MechanicalView
DQMHistoSizes: changed ( 1002.4 ): 0.059 KiB JetMET/SUSYDQM
Checked 224 log files, 45 edm output root files, 54 DQM output files
TriggerResults: found differences in 2 / 53 workflows

tvami · 2022-10-19T11:57:02Z

+alca

failures are unrelated and known

cmsbuild · 2022-10-19T12:00:36Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_4_X IBs (but tests are reportedly failing) and once validation in the development release cycle CMSSW_12_6_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

rappoccio · 2022-10-19T14:25:15Z

I'm a bit confused, the main PR has changes to CondFormats/JetMETObjects/src/SimpleJetCorrector.cc, but the 12_4 and 12_5 (#39751) don't have these. Is this expected and known?

francescobrivio · 2022-10-19T14:26:34Z

I'm a bit confused, the main PR has changes to CondFormats/JetMETObjects/src/SimpleJetCorrector.cc, but the 12_4 and 12_5 (#39751) don't have these. Is this expected and known?

Hi Sal, please see discussion at #39519 (comment)

perrotta · 2022-10-21T07:08:48Z

Configuration/AlCa/python/autoAlca.py

@@ -51,7 +51,7 @@ def buildList(pdList, matrix):

 # Update the lists anytime a new PD is added to the matrix
 autoAlca = { 'allForPrompt'         : buildList(['Charmonium', 'Commissioning', 'DoubleMuParked', 'DoubleMuon', 'DoubleMuonLowMass', 'EGamma', 'HLTPhysics', 'HcalNZS', 'JetHT', 'MET', 'MinimumBias', 'MuOnia', 'MuOniaParked', 'NoBPTX', 'SingleMuon', 'ZeroBias'], AlCaRecoMatrix),


Not relevant for this PR, but I am noticing that the 'allForPrompt' list is differenn here and in the master or 12_5_X
@tvami is it intended? (Perhaps I just forgot when it branched away...)

@tvami (or @cms-sw/alca-l2) could you please check and answer this, before it gets forgotten, fiven that the PR is now merged?

hi @perrotta sorry for the delay! we'll take a look asap!
(wrong Tamas tagged btw :D )

Hi @perrotta , this should indeed have been backported from PR #38951 . I will submit the backport today. Thank you very much for spotting it!

I just submitted the backport: #39837

rappoccio · 2022-10-21T13:15:51Z

+1

The DQM tests look unrelated to this PR.

rappoccio · 2022-10-21T13:50:19Z

merge

Backport to CMSSW_12_4_X

0ff3d66

cmsbuild added this to the CMSSW_12_4_X milestone Oct 18, 2022

cmsbuild added alca-pending orp-pending pending-signatures tests-pending labels Oct 18, 2022

purbita-rahman changed the title ~~Backport to CMSSW_12_4_X~~ [12_4_X] Remove hardcoded trigger bits from Lumi ALCARECOs Oct 18, 2022

cmsbuild added tests-started and removed tests-pending labels Oct 18, 2022

cmsbuild added the backport-ok label Oct 18, 2022

cmsbuild added tests-pending and removed tests-started labels Oct 18, 2022

cmsbuild added tests-started and removed tests-pending labels Oct 18, 2022

cmsbuild added tests-rejected and removed tests-started labels Oct 18, 2022

cmsbuild added tests-started and removed tests-rejected labels Oct 18, 2022

cmsbuild added tests-rejected and removed tests-started labels Oct 19, 2022

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Oct 19, 2022

rappoccio mentioned this pull request Oct 19, 2022

[12_5_X] Remove hardcoded trigger bits from Lumi ALCARECOs #39751

Closed

perrotta reviewed Oct 21, 2022

View reviewed changes

cmsbuild added orp-approved and removed orp-pending labels Oct 21, 2022

cmsbuild merged commit 031f166 into cms-sw:CMSSW_12_4_X Oct 21, 2022

malbouis mentioned this pull request Oct 24, 2022

Remove hardcoded trigger bits from AlCaRecos and include it as a key in the alcareco trigger bits tag. cms-AlCaDB/AlCaTools#70

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[12_4_X] Remove hardcoded trigger bits from Lumi ALCARECOs #39752

[12_4_X] Remove hardcoded trigger bits from Lumi ALCARECOs #39752

purbita-rahman commented Oct 18, 2022 •

edited

cmsbuild commented Oct 18, 2022 •

edited

ChrisMisan commented Oct 18, 2022

ChrisMisan commented Oct 18, 2022

ChrisMisan commented Oct 18, 2022

ChrisMisan commented Oct 18, 2022

ChrisMisan commented Oct 18, 2022

ChrisMisan commented Oct 18, 2022

cmsbuild commented Oct 18, 2022

tvami commented Oct 18, 2022

cmsbuild commented Oct 19, 2022

tvami commented Oct 19, 2022

cmsbuild commented Oct 19, 2022

rappoccio commented Oct 19, 2022

francescobrivio commented Oct 19, 2022

perrotta Oct 21, 2022 •

edited

perrotta Oct 24, 2022 •

edited

francescobrivio Oct 24, 2022

malbouis Oct 24, 2022

malbouis Oct 24, 2022

rappoccio commented Oct 21, 2022

rappoccio commented Oct 21, 2022

		@@ -51,7 +51,7 @@ def buildList(pdList, matrix):

		# Update the lists anytime a new PD is added to the matrix
		autoAlca = { 'allForPrompt' : buildList(['Charmonium', 'Commissioning', 'DoubleMuParked', 'DoubleMuon', 'DoubleMuonLowMass', 'EGamma', 'HLTPhysics', 'HcalNZS', 'JetHT', 'MET', 'MinimumBias', 'MuOnia', 'MuOniaParked', 'NoBPTX', 'SingleMuon', 'ZeroBias'], AlCaRecoMatrix),

[12_4_X] Remove hardcoded trigger bits from Lumi ALCARECOs #39752

[12_4_X] Remove hardcoded trigger bits from Lumi ALCARECOs #39752

Conversation

purbita-rahman commented Oct 18, 2022 • edited

PR description:

cmsbuild commented Oct 18, 2022 • edited

ChrisMisan commented Oct 18, 2022

ChrisMisan commented Oct 18, 2022

ChrisMisan commented Oct 18, 2022

ChrisMisan commented Oct 18, 2022

ChrisMisan commented Oct 18, 2022

ChrisMisan commented Oct 18, 2022

cmsbuild commented Oct 18, 2022

Unit Tests

Comparison Summary

tvami commented Oct 18, 2022

cmsbuild commented Oct 19, 2022

Unit Tests

Comparison Summary

tvami commented Oct 19, 2022

cmsbuild commented Oct 19, 2022

rappoccio commented Oct 19, 2022

francescobrivio commented Oct 19, 2022

perrotta Oct 21, 2022 • edited

Choose a reason for hiding this comment

perrotta Oct 24, 2022 • edited

Choose a reason for hiding this comment

francescobrivio Oct 24, 2022

Choose a reason for hiding this comment

malbouis Oct 24, 2022

Choose a reason for hiding this comment

malbouis Oct 24, 2022

Choose a reason for hiding this comment

rappoccio commented Oct 21, 2022

rappoccio commented Oct 21, 2022

purbita-rahman commented Oct 18, 2022 •

edited

cmsbuild commented Oct 18, 2022 •

edited

perrotta Oct 21, 2022 •

edited

perrotta Oct 24, 2022 •

edited