Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update GTs for 2023 #40445

Merged
merged 3 commits into from Jan 11, 2023
Merged

Conversation

francescobrivio
Copy link
Contributor

@francescobrivio francescobrivio commented Jan 7, 2023

PR description:

This PR updates the GTs for 2023:

GT differences:

PR validation:

Successfully tested with:
runTheMatrix.py -l 136.731,138.5,138.4,139.001,136.899,12034.0,11634.0,7.23,312.0,12834.0,20834.0 -j10 --ibeos

Backport:

Not a backport.
A backport to 12_6_X with data GTs only will be opened soon.

@francescobrivio
Copy link
Contributor Author

test parameters:

  • workflows = 12034.0,7.23,12834.0,136.899

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 7, 2023

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40445/33597

  • This PR adds an extra 12KB to repository

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 7, 2023

A new Pull Request was created by @francescobrivio for master.

It involves the following packages:

  • Configuration/AlCa (alca)

@malbouis, @yuanchao, @cmsbuild, @saumyaphor4252, @francescobrivio, @ChrisMisan, @tvami can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @mmusich, @fabiocos, @tocheng this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

@missirol
Copy link
Contributor

missirol commented Jan 7, 2023

I did not find any relval in the matrix that consumes the postEE GT

That should be wf 13634.0 (see #40151 (comment)).

@francescobrivio
Copy link
Contributor Author

That should be wf 13634.0 (see #40151 (comment)).

Thanks Marino!
Indeed I ran runTheMatrix.py -l 13634.0 --what upgrade -j 8 privately and it ended successfully.
@cms-sw/alca-l2 I don't think we need to re-trigger tests just for this, in case you disagree feel free to re-trigger them with the addition of this wf.

@francescobrivio
Copy link
Contributor Author

test parameters:
*

@francescobrivio
Copy link
Contributor Author

test parameters:

  • workflows = 12034.0,7.23,12834.0,136.899,13634.0

@francescobrivio
Copy link
Contributor Author

@cmsbuild please test

I realized the tests hadn't started yet, so I added the postEE MC wfs as well!

@missirol
Copy link
Contributor

missirol commented Jan 7, 2023

To run wf 13634.0, I think one also needs to set relvals_opt like in #40151 (comment):
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-relvals/28542/consoleText

Traceback (most recent call last):
File "/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/40445/29826/CMSSW_13_0_X_2023-01-07-1100/bin/el8_amd64_gcc11/runTheMatrix.py", line 623, in
ret = runSelected(opt)
File "/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/40445/29826/CMSSW_13_0_X_2023-01-07-1100/bin/el8_amd64_gcc11/runTheMatrix.py", line 31, in runSelected
if len(undefSet)>0: raise ValueError('Undefined workflows: '+', '.join(map(str,list(undefSet))))
ValueError: Undefined workflows: 13634.0

(Obviously, there are also other errors from RelVals and AddOnTests.)

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 7, 2023

-1

Failed Tests: RelVals AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-58cd4a/29826/summary.html
COMMIT: 71b2028
CMSSW: CMSSW_13_0_X_2023-01-07-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/40445/29826/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

ValueError: Undefined workflows: 13634.0

----- Begin Fatal Exception 07-Jan-2023 20:00:02 CET-----------------------
An exception of category 'NoProxyException' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Prefetching for module DQMMessageLogger/'DQMMessageLogger'
   [3] Prefetching for module LogErrorHarvester/'logErrorHarvester'
   [4] Prefetching for module DeepFlavourJetTagsProducer/'pfDeepCMVAJetTags'
   [5] Prefetching for module DeepCMVATagInfoProducer/'pfDeepCMVATagInfos'
   [6] Calling method for EventSetup module ElectronTaggerESProducer/'softPFElectronComputer'
Exception Message:
No data of type "GBRForest" with label "btag_SoftPFElectron_BDT" in record "GBRWrapperRcd"
 Please add an ESSource or ESProducer to your job which can deliver this data.
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 07-Jan-2023 20:06:38 CET-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'dqmofflineOnPAT_1_step'
   [2] Prefetching for module JetAnalyzer/'jetDQMAnalyzerAk8PFPUPPICleanedMiniAOD'
   [3] Prefetching for module PATJetSelector/'slimmedJetsAK8'
   [4] Prefetching for module PATJetUpdater/'updatedPatJetsTransientCorrectedSlimmedAK8DeepTags'
   [5] Prefetching for module PATJetUpdater/'updatedPatJetsSlimmedAK8DeepTags'
   [6] Prefetching for module PATJetSlimmer/'slimmedJetsAK8NoDeepTags'
   [7] Prefetching for module JetSubstructurePacker/'packedPatJetsAK8'
   [8] Prefetching for module PATJetSelector/'selectedPatJetsAK8Puppi'
   [9] Prefetching for module PATJetProducer/'patJetsAK8Puppi'
   [10] Prefetching for module JetTagProducer/'pfCombinedInclusiveSecondaryVertexV2BJetTagsAK8Puppi'
   [11] Calling method for EventSetup module CandidateCombinedSecondaryVertexESProducer/'candidateCombinedSecondaryVertexV2Computer'
   [12] While getting dependent Record from Record JetTagComputerRecord
Exception Message:
No "BTauGenericMVAJetTagComputerRcd" record found in the EventSetup.

 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 07-Jan-2023 20:10:17 CET-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'dqmofflineOnPAT_1_step'
   [2] Prefetching for module JetAnalyzer/'jetDQMAnalyzerAk8PFPUPPICleanedMiniAOD'
   [3] Prefetching for module PATJetSelector/'slimmedJetsAK8'
   [4] Prefetching for module PATJetUpdater/'updatedPatJetsTransientCorrectedSlimmedAK8DeepTags'
   [5] Prefetching for module PATJetUpdater/'updatedPatJetsSlimmedAK8DeepTags'
   [6] Prefetching for module PATJetSlimmer/'slimmedJetsAK8NoDeepTags'
   [7] Prefetching for module JetSubstructurePacker/'packedPatJetsAK8'
   [8] Prefetching for module PATJetSelector/'selectedPatJetsAK8Puppi'
   [9] Prefetching for module PATJetProducer/'patJetsAK8Puppi'
   [10] Prefetching for module JetTagProducer/'pfCombinedInclusiveSecondaryVertexV2BJetTagsAK8Puppi'
   [11] Calling method for EventSetup module CandidateCombinedSecondaryVertexESProducer/'candidateCombinedSecondaryVertexV2Computer'
   [12] While getting dependent Record from Record JetTagComputerRecord
Exception Message:
No "BTauGenericMVAJetTagComputerRcd" record found in the EventSetup.

 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------
Expand to see more relval errors ...

AddOn Tests

----- Begin Fatal Exception 07-Jan-2023 19:56:23 CET-----------------------
An exception of category 'NoProxyException' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 2
   [1] Running path 'FEVTDEBUGHLToutput_step'
   [2] Prefetching for module PoolOutputModule/'FEVTDEBUGHLToutput'
   [3] Prefetching for module LogErrorHarvester/'logErrorHarvester'
   [4] Prefetching for module DeepFlavourJetTagsProducer/'pfDeepCMVAJetTags'
   [5] Prefetching for module DeepCMVATagInfoProducer/'pfDeepCMVATagInfos'
   [6] Calling method for EventSetup module ElectronTaggerESProducer/'softPFElectronComputer'
Exception Message:
No data of type "GBRForest" with label "btag_SoftPFElectron_BDT" in record "GBRWrapperRcd"
 Please add an ESSource or ESProducer to your job which can deliver this data.
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 07-Jan-2023 19:55:23 CET-----------------------
An exception of category 'NoProxyException' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 2 stream: 1
   [1] Running path 'FEVTDEBUGHLToutput_step'
   [2] Prefetching for module PoolOutputModule/'FEVTDEBUGHLToutput'
   [3] Prefetching for module LogErrorHarvester/'logErrorHarvester'
   [4] Prefetching for module DeepFlavourJetTagsProducer/'pfDeepCMVAJetTags'
   [5] Prefetching for module DeepCMVATagInfoProducer/'pfDeepCMVATagInfos'
   [6] Calling method for EventSetup module MuonTaggerESProducer/'softPFMuonComputer'
Exception Message:
No data of type "GBRForest" with label "btag_SoftPFMuon_BDT" in record "GBRWrapperRcd"
 Please add an ESSource or ESProducer to your job which can deliver this data.
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 07-Jan-2023 19:54:33 CET-----------------------
An exception of category 'NoProxyException' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 3
   [1] Running path 'FEVTDEBUGHLToutput_step'
   [2] Prefetching for module PoolOutputModule/'FEVTDEBUGHLToutput'
   [3] Prefetching for module LogErrorHarvester/'logErrorHarvester'
   [4] Prefetching for module DeepFlavourJetTagsProducer/'pfDeepCMVAJetTags'
   [5] Prefetching for module DeepCMVATagInfoProducer/'pfDeepCMVATagInfos'
   [6] Calling method for EventSetup module MuonTaggerESProducer/'softPFMuonComputer'
Exception Message:
No data of type "GBRForest" with label "btag_SoftPFMuon_BDT" in record "GBRWrapperRcd"
 Please add an ESSource or ESProducer to your job which can deliver this data.
----- End Fatal Exception -------------------------------------------------
Expand to see more addon errors ...

@francescobrivio
Copy link
Contributor Author

Ok based on the errors I think we still need to add Run3 era modifiers in:

@yuanchao could you take a look at this ~quickly?

@yuanchao
Copy link
Contributor

yuanchao commented Jan 9, 2023

@francescobrivio Yes, for the first two modifications, we can try turn useCondDB off. As to candidateCombinedSecondaryVertexV2Computer, it's the one becomes deprecated in run3. We need to change it to Deep b-taggers.

@francescobrivio
Copy link
Contributor Author

@francescobrivio Yes, for the first two modifications, we can try turn useCondDB off. As to candidateCombinedSecondaryVertexV2Computer, it's the one becomes deprecated in run3. We need to change it to Deep b-taggers.

Thanks a lot Yuan! I'll give it a try with a PR today!

@tvami
Copy link
Contributor

tvami commented Jan 10, 2023

+alca

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

@perrotta
Copy link
Contributor

+1

@perrotta
Copy link
Contributor

@francescobrivio IBs show an error, so far in wf 11634.15, likely coming from the GTs updated with this PR:

----- Begin Fatal Exception 12-Jan-2023 08:01:24 CET-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 3 stream: 3
   [1] Running path 'dqmoffline_3_step'
   [2] Prefetching for module NanoAODDQM/'nanoDQMMC'
   [3] Prefetching for module CandMCMatchTableProducer/'boostedTauMCTable'
   [4] Prefetching for module PATObjectCrossLinker/'linkedObjects'
   [5] Prefetching for module PATJetRefSelector/'finalJetsPuppi'
   [6] Prefetching for module PATJetUserDataEmbedder/'updatedJetsPuppiWithUserData'
   [7] Prefetching for module PATJetUpdater/'updatedJetsPuppi'
   [8] Prefetching for module PATJetSelector/'selectedUpdatedPatJetsAK4PFPUPPIFinal'
   [9] Prefetching for module PATJetUpdater/'updatedPatJetsTransientCorrectedAK4PFPUPPIFinal'
   [10] Prefetching for module JetTagProducer/'pfCombinedInclusiveSecondaryVertexV2BJetTagsAK4PFPUPPIFinal'
   [11] Calling method for EventSetup module CandidateCombinedSecondaryVertexESProducer/'candidateCombinedSecondaryVertexV2Computer'
   [12] While getting dependent Record from Record JetTagComputerRecord
Exception Message:
No "BTauGenericMVAJetTagComputerRcd" record found in the EventSetup.

 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

Could you please have a look?

@francescobrivio
Copy link
Contributor Author

Could you please have a look?

Yes, seems related to this PR. I'm checking it now!

@tvami
Copy link
Contributor

tvami commented Jan 12, 2023

Maybe #40485 (comment) will resolve this

@francescobrivio
Copy link
Contributor Author

Maybe #40485 (comment) will resolve this

Unfortunately I don't think so. That PR is modifying the softLepton producers, while this error comes from a jet sequence. I'm still trying to figure out exactly where it's colled so I can add the proper modifier for Run3...

@missirol
Copy link
Contributor

I'm still trying to figure out exactly where it's colled so I can add the proper modifier for Run3...

I guess it comes from (all) the lines with "CSVV2" in
https://github.com/cms-sw/cmssw/blob/ac8ac2424137e35f9ca819a219c675593475b6f6/PhysicsTools/NanoAOD/python/custom_jme_cff.py

Btw, #40485 (comment) can be aborted since the failing wf is not 11634.1 (it is 11634.15).

@AnnikaStein
Copy link
Contributor

Given that cross-linking between taus (mentioned for the boostedTauMCTable in the exception message in #40445 (comment)) and jets is also performed in jetsAK4_Puppi_cff.py and the same for CHS, it seems a modifier to exclude CSVV2 is also necessary for the lines here https://github.com/cms-sw/cmssw/blob/master/PhysicsTools/NanoAOD/python/jetsAK4_Puppi_cff.py#L85 and here https://github.com/cms-sw/cmssw/blob/master/PhysicsTools/NanoAOD/python/jetsAK4_CHS_cff.py#L127

For FatJets, SubJets there is also https://github.com/cms-sw/cmssw/blob/master/PhysicsTools/NanoAOD/python/jetsAK8_cff.py#L95 and https://github.com/cms-sw/cmssw/blob/master/PhysicsTools/NanoAOD/python/jetsAK8_cff.py#L217

@tvami
Copy link
Contributor

tvami commented Jan 12, 2023

hi @AnnikaStein should we have this fixed in the PR40485 ?

@perrotta
Copy link
Contributor

hi @AnnikaStein should we have this fixed in the PR40485 ?

Tests with 11634.15 are currently running in #40485

@AnnikaStein
Copy link
Contributor

hi @AnnikaStein should we have this fixed in the PR40485 ?

Yes, let me try to add also the proposed changes to NanoAOD tables such that CSVV2 is only included for run2 era modifiers.

@francescobrivio francescobrivio deleted the alca-update_GTs_2023 branch January 13, 2023 09:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants