[12_2_X] Add Run3 relval GTs and use fixed snapshot in online GTs #37670
Conversation
|
A new Pull Request was created by @francescobrivio for CMSSW_12_2_X. It involves the following packages:
@malbouis, @yuanchao, @cmsbuild, @missirol, @Martin-Grunewald, @francescobrivio, @tvami can you please review it and eventually sign? Thanks. cms-bot commands are listed here |
|
@cmsbuild , please test |
|
backport of #37171 |
|
-1 Failed Tests: RelVals-INPUT RelVals-INPUT
Comparison SummarySummary:
|
|
Can we progress on this PR, so that the IB HLT-validation and TSG tests work again? The test failure can not be related to this PR. In general the use case from the TSG side for these two special GTs is to run the (more recent and developing) HLT menus on data taken with an older HLT menu. The goal is NOT to reproduce the old HLT from the time the data was taken, but so see the performance of the new HLT menu, in turn using new calibrations etc, on that old data (to be replaced by real Run3 collisions once we have them). Of course, if 12_2 is no longer needed, then we could let it rot but then it should be removed from IB/relval testing, ie, |
|
Pull request #37670 was updated. @malbouis, @yuanchao, @cmsbuild, @missirol, @Martin-Grunewald, @francescobrivio, @tvami can you please check and sign again. |
|
test parameters:
|
|
@cmsbuild please test |
Hi Martin, sorry for the delay I was offline for a couple of days, it should be fixed now. We would like to test in 12_2_X IBs if everything is ok like this, if so we will update also 12_3_X and 12_4_X with the same policy. @perrotta @qliphy I know the forward porting is not the standard procedure, but in this case using the 12_2_X IBs is perfect to reproduce the boost version change between releases, I hope that is fine with you 😄 |
francescobrivio
changed the title
|
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-79a9f7/24365/summary.html Comparison SummaryThere are some workflows for which there are errors in the baseline: Summary:
|
Sorry, but does this |
Hi Martin, yes that is correct. Please note that for the relval GTs the snapshot has always been frozen, see e.g. the current 123X relval GTs: |
|
OK, thanks for the clarification and the workflow to update. |
|
+alca
|
cmsbuild
added
alca-approved
fully-signed
and removed
alca-pending
pending-signatures
labels
|
This pull request is fully signed and it will be integrated in one of the next CMSSW_12_2_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_4_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2) |
Except is not new at all :D |
Is anybody able to trace where do those extra lines in the logs come from? |
|
Isnt that reffering to this Most things I see here are not related to this PR |
|
Just a guess: maybe it has to do with the wf that was failing in IB, and passes now thanks to this PR, i.e. That wf does not run HLT, and I don't know if those log messages are to be expected. |
If that wf does not run HLT, why are we running the HLT tracking monitoring in it? That is producing a gazillion of product not found warnings... |
|
Good question for DQM? @cms-sw/dqm-l2 |
Actually, the question might be better directed to who designed, or maintains, that wf. In any case, one way to fix this is [*]. I can't judge if that's the correct fix, that's left to the experts. It's clear that this issue is not introduced by this PR, and it likely needs to be fixed in Arguably, this PR should not be delayed because of it. [*] diff --git a/Configuration/PyReleaseValidation/python/relval_steps.py b/Configuration/PyReleaseValidation/python/relval_steps.py
index d7a461e1bda..2dde975c0c4 100644
--- a/Configuration/PyReleaseValidation/python/relval_steps.py
+++ b/Configuration/PyReleaseValidation/python/relval_steps.py
@@ -2738,7 +2738,7 @@ steps['ALCARECOEXPR3']=merge([{'-s':'ALCAOUTPUT:SiPixelCalZeroBias+SiStripCalZer
'--triggerResultsProcess': 'RECO',
'--customise':'Configuration/DataProcessing/RecoTLR.customiseExpress'},steps['RECODR3']])
-steps['ALCARECOPROMPTR3']=merge([{'-s':'RAW2DIGI,L1Reco,RECO,ALCA:SiStripCalZeroBias+SiStripCalMinBias+TkAlMinBias+HcalCalHO+HcalCalIterativePhiSym+HcalCalHBHEMuonProducerFilter+HcalCalIsoTrkProducerFilter,DQM',
+steps['ALCARECOPROMPTR3']=merge([{'-s':'RAW2DIGI,L1Reco,RECO,ALCA:SiStripCalZeroBias+SiStripCalMinBias+TkAlMinBias+HcalCalHO+HcalCalIterativePhiSym+HcalCalHBHEMuonProducerFilter+HcalCalIsoTrkProducerFilter,DQM:@standardDQMFakeHLT',
'--conditions':'auto:run3_data_prompt',
'--scenario':'pp',
'--era':'Run3', |
|
@perrotta, the issue of spurious warnings in wf Do you think this PR can now be integrated? |
is there any easy way to scan from where these come from? |
|
@mmusich , please check the https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_12_2_X_2022-05-01-0000+79a9f7/49980/validateJR/logRootQA.log log file for details |
I think this is indeed probably the cause of the additions (from the log above, thanks @smuzaffar !):
|
|
Thank you @smuzaffar : it is not an easy read, but I also think that it can confirm that the extra lines in the log mostly come from 138.4. According to that link the extra lines in that workflow log are 2712, while #37670 (comment) says that overall the extra lines are 940. It means that all other log comprehensively remove O(1800) lines. I think they could be the oned detailed in the bottom of https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_12_2_X_2022-05-01-0000+79a9f7/49980/validateJR/logRootQA.log where "You removed..." is written. But I don't think that any more detailed debug of the issue is needed ;-) Conclusion: what @missirol suggested in #37670 (comment) makes sense, and we can merge this PR, so that next IB tests in 12_2_X will be clean. |
|
+1
|
PR description:
Backport of #37171
As reported in #37653 (comment) (and later in the AlCaDB CMS Talk forum by @Martin-Grunewald), since datataking switched to CMSSW_12_3_X (and the boost version in use is now 1_78) the HLT tests in the 12_2_X IBs are failing because they are still using the
auto:run3_hltandauto:run3_data_promptGTs (which have an infinite snapshot).This PR introduces, for the 12_2_X cycle, the same "relval" autoCond entries for run3 data there were already introduced in 12_3_X and 12_4_X, namely:
auto:run3_hlt_relval--> GT: 122X_dataRun3_HLT_relval_v2auto:run3_data_relval--> GT: 122X_dataRun3_relval_v2Following @missirol suggestion I also updated
autoCondHLT.pyto use the newly created GTs.Edit:
After some discussion in @cms-sw/alca-l2 we decided it is better NOT to have "open-ended" GTs in the release as they can cause all sort of troubles in the IBs and also possibly reproducibility issues.
So I updated all the online GTs (HLT/Express/Prompt) to a "frozen" version, i.e. with a fixed snapshot at the time of the last run processed with 12_2_X in Tier0 (as announced in this CMSTalk post): Run 350279 - Snapshot:
2022-04-12 15:15:00Differences wrt the hlt and offline Run3 data GTs are:
Run 3 HLT RelVals
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/122X_dataRun3_HLT_v4/122X_dataRun3_HLT_relval_v2
Run 3 data RelVals
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/122X_dataRun3_v3/122X_dataRun3_relval_v2
Run 3 HLT
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/122X_dataRun3_HLT_v4/122X_dataRun3_HLT_frozen_v1
Run 3 Express
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/122X_dataRun3_Express_v3/122X_dataRun3_Express_frozen_v1
Run 3 Prompt
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/122X_dataRun3_Prompt_v3/122X_dataRun3_Prompt_frozen_v1
PR validation:
Code compiles and I run the same HLT test that is running in the IBs (thanks @missirol for the recipe):
As expected this produces:
Proving that the updates are correct.
Backport
backport of #37171