Mask old b-taggers and keep only DNN taggers with Era modifier for run3. #39808

yuanchao · 2022-10-21T08:19:14Z

PR description:

Regarding the AlCa issue #58 (cms-AlCaDB/AlCaTools#58), old b-tagging GT records are request to be removed (starting) from 12_1_X (and so on). Due to the dependency on "BTauGenericMVAJetTagComputerRcd" and "JetTagComputerRecord", the corresponding b-tagging computers need to be masked in run3. As DNN taggers will be the only supported ones in run3, this PR masks the old taggers with Era modifier and keeps only DNN taggers like DeepCSV and particleNet. (track counting and jet probability are kept for reference as requested)

PR validation:

PR tested in local with runTheMatrix.py processes. However, the standard run 2 validation sequences will fail due to missing b-tagger objects.

This PR is for run 3. Not a backport and no backport planed.

cmsbuild · 2022-10-21T08:26:34Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39808/32679

This PR adds an extra 12KB to repository

cmsbuild · 2022-10-21T08:26:53Z

A new Pull Request was created by @yuanchao (Yuan CHAO) for master.

It involves the following packages:

RecoBTag/Configuration (reconstruction)

@cmsbuild, @mandrenguyen, @clacaputo can you please review it and eventually sign? Thanks.
@AlexDeMoor, @emilbols, @JyothsnaKomaragiri, @AnnikaStein, @missirol, @andrzejnovak, @demuller this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

mandrenguyen · 2022-10-21T09:43:54Z

please test

cmsbuild · 2022-10-21T10:56:04Z

-1

Failed Tests: RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3763cc/28420/summary.html
COMMIT: 807ac22
CMSSW: CMSSW_12_6_X_2022-10-20-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39808/28420/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

139.001

----- Begin Fatal Exception 21-Oct-2022 12:04:43 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 346512 lumi: 250 event: 243048393 stream: 0
   [1] Running path 'dqmoffline_1_step'
   [2] Prefetching for module MiniAODSVAnalyzer/'bTagSVDQM'
   [3] Calling method for module PATJetProducer/'patJetsSVInfo'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: edm::AssociationVector<edm::RefToBaseProd<reco::Jet>,std::vector<float>,edm::RefToBase<reco::Jet>,unsigned int,edm::helper::AssociationIdenticalKeyReference>
Looking for module label: pfJetBProbabilityBJetTags
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

312.0

----- Begin Fatal Exception 21-Oct-2022 12:08:10 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'dqmoffline_1_step'
   [2] Prefetching for module MuonMiniAOD/'MuonMiniAOD'
   [3] Prefetching for module PATMuonSlimmer/'slimmedMuons'
   [4] Prefetching for module PATMuonSelector/'selectedPatMuons'
   [5] Calling method for module PATMuonProducer/'patMuons'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: edm::AssociationVector<edm::RefToBaseProd<reco::Jet>,std::vector<float>,edm::RefToBase<reco::Jet>,unsigned int,edm::helper::AssociationIdenticalKeyReference>
Looking for module label: pfCombinedInclusiveSecondaryVertexV2BJetTags
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

13234.0

----- Begin Fatal Exception 21-Oct-2022 12:15:13 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'dqmoffline_1_step'
   [2] Prefetching for module MiniAODSVAnalyzer/'bTagSVDQM'
   [3] Calling method for module PATJetProducer/'patJetsSVInfo'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: edm::AssociationVector<edm::RefToBaseProd<reco::Jet>,std::vector<float>,edm::RefToBase<reco::Jet>,unsigned int,edm::helper::AssociationIdenticalKeyReference>
Looking for module label: pfJetBProbabilityBJetTags
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

AlexDeMoor · 2022-10-21T12:54:46Z

Hi @yuanchao , what about JP and DeepJet ?
Those taggers are still used by the BTV group. DeepJet is still the sota model and will be kept at the beginning of Run3 and JP is used for calibration

yuanchao · 2022-10-24T11:35:14Z

@AlexDeMoor Thank you very much for the comment. Originally I was told that DNN taggers will be the only supported ones in Run 3. Surely I can add back the JP tagger in the era modifier. As to DeepJet, it's not in the original RecoBTag_cff.py configuration file. This PR doesn't touch that.

yuanchao · 2022-10-24T11:53:02Z

@AlexDeMoor One more thing. Does JP depend on "JetTagComputerRecord" GT?

tvami · 2022-10-27T15:33:23Z

assign alca

cmsbuild · 2022-10-27T15:33:43Z

New categories assigned: alca

@yuanchao,@francescobrivio,@malbouis,@saumyaphor4252,@tvami,@ChrisMisan you have been requested to review this Pull request/Issue and eventually sign? Thanks

AlexDeMoor · 2022-10-27T20:40:25Z

Thank you for the information @yuanchao . Kindly ping @soureek and @johnalison . Do you have the answer about the GT ("Does JP depend on "JetTagComputerRecord" GT?")

soureek · 2022-11-03T12:32:39Z

@AlexDeMoor One more thing. Does JP depend on "JetTagComputerRecord" GT?

As mentioned by Daniel Bloch in the email thread, we don't require this.

tvami · 2022-11-11T11:55:28Z

@cmsbuild , please test with #40043

cmsbuild · 2022-11-11T13:22:44Z

-1

Failed Tests: RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3763cc/28973/summary.html
COMMIT: 807ac22
CMSSW: CMSSW_12_6_X_2022-11-11-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39808/28973/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

139.001

----- Begin Fatal Exception 11-Nov-2022 13:30:15 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 346512 lumi: 250 event: 243048393 stream: 0
   [1] Running path 'dqmoffline_1_step'
   [2] Prefetching for module MiniAODSVAnalyzer/'bTagSVDQM'
   [3] Calling method for module PATJetProducer/'patJetsSVInfo'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: edm::AssociationVector<edm::RefToBaseProd<reco::Jet>,std::vector<float>,edm::RefToBase<reco::Jet>,unsigned int,edm::helper::AssociationIdenticalKeyReference>
Looking for module label: pfJetBProbabilityBJetTags
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

312.0

----- Begin Fatal Exception 11-Nov-2022 13:33:10 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'dqmoffline_1_step'
   [2] Prefetching for module MuonMiniAOD/'MuonMiniAOD'
   [3] Prefetching for module PATMuonSlimmer/'slimmedMuons'
   [4] Prefetching for module PATMuonSelector/'selectedPatMuons'
   [5] Calling method for module PATMuonProducer/'patMuons'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: edm::AssociationVector<edm::RefToBaseProd<reco::Jet>,std::vector<float>,edm::RefToBase<reco::Jet>,unsigned int,edm::helper::AssociationIdenticalKeyReference>
Looking for module label: pfCombinedInclusiveSecondaryVertexV2BJetTags
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

13234.0

----- Begin Fatal Exception 11-Nov-2022 13:39:45 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'dqmoffline_1_step'
   [2] Prefetching for module MiniAODSVAnalyzer/'bTagSVDQM'
   [3] Calling method for module PATJetProducer/'patJetsSVInfo'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: edm::AssociationVector<edm::RefToBaseProd<reco::Jet>,std::vector<float>,edm::RefToBase<reco::Jet>,unsigned int,edm::helper::AssociationIdenticalKeyReference>
Looking for module label: pfJetBProbabilityBJetTags
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

cmsbuild · 2022-11-18T16:46:04Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3763cc/29114/summary.html
COMMIT: 8764b76
CMSSW: CMSSW_12_6_X_2022-11-18-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39808/29114/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 1108 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3417167
DQMHistoTests: Total failures: 4278
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3412867
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 206 log files, 48 edm output root files, 48 DQM output files
TriggerResults: no differences found

tvami · 2022-11-18T16:54:02Z

+alca

changes are in BTV related folders which is expected

AlexDeMoor · 2022-11-22T13:16:07Z

After looking at the last test (https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3763cc/29114/summary.html), it seems we have a lot of failings coming from the BTV HLT. I kindly ping @johnalison , @NiclasEich and @marco-link for their feedback on this part 👀

clacaputo · 2022-12-02T09:18:21Z

please test

refreshing the results

cmsbuild · 2022-12-02T13:22:24Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3763cc/29418/summary.html
COMMIT: 8764b76
CMSSW: CMSSW_13_0_X_2022-12-01-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39808/29418/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 1274 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3421159
DQMHistoTests: Total failures: 3935
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3417202
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 206 log files, 158 edm output root files, 48 DQM output files
TriggerResults: no differences found

mandrenguyen · 2022-12-08T15:26:23Z

After looking at the last test (https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3763cc/29114/summary.html), it seems we have a lot of failings coming from the BTV HLT. I kindly ping @johnalison , @NiclasEich and @marco-link for their feedback on this part 👀

Hello, are we still waiting for feedback from BTV HLT? @cms-sw/btv-pog-l2

tvami · 2022-12-08T18:30:12Z

hi @mandrenguyen I had a private communication with Soureek et al, and they said:
"We are discussing a bit with the developers how to simplify/update the access of the DNN tagger information within CMSSW. However, I'm sorry to say that this discussion will need a bit more time to come to a final conclusion leaving no loose ends. We expect to have a concrete plan by end-January, 2023. Hope this works with you."

I'd say we should let this PR in, and then let BTV remove the remaining dependencies for the HLT folder, what do you think?

mandrenguyen · 2022-12-08T21:23:58Z

+reconstruction

cmsbuild · 2022-12-08T21:24:13Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

rappoccio · 2022-12-09T14:39:58Z

+1

cmsbuild added this to the CMSSW_12_6_X milestone Oct 21, 2022

cmsbuild added code-checks-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Oct 21, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 21, 2022

cmsbuild added tests-started and removed tests-pending labels Oct 21, 2022

cmsbuild added tests-rejected and removed tests-started labels Oct 21, 2022

cmsbuild added the alca-pending label Oct 27, 2022

cmsbuild mentioned this pull request Nov 9, 2022

HLT_Mu17_Photon30_IsoCaloId_v trigger filters backport to CMSSW_12_4_X #40034

Closed

cmsbuild added requires-external tests-started and removed tests-rejected labels Nov 11, 2022

cmsbuild added tests-approved and removed tests-started labels Nov 18, 2022

cmsbuild added alca-approved and removed alca-pending labels Nov 18, 2022

slava77 mentioned this pull request Nov 19, 2022

automatic finding of edm and pure root nano branches cms-sw/cms-bot#1872

Merged

cmsbuild modified the milestones: CMSSW_12_6_X, CMSSW_13_0_X Nov 24, 2022

cmsbuild added tests-started and removed tests-approved labels Dec 2, 2022

cmsbuild added tests-approved and removed tests-started labels Dec 2, 2022

cmsbuild added fully-signed reconstruction-approved and removed reconstruction-pending pending-signatures labels Dec 8, 2022

cmsbuild added orp-approved and removed orp-pending labels Dec 9, 2022

cmsbuild merged commit 1a927fc into cms-sw:master Dec 9, 2022

cmsbuild mentioned this pull request Dec 10, 2022

ROOT 6 with LLVM 13: Updated root to tip of branch master cms-sw/cmsdist#8223

Merged

francescobrivio mentioned this pull request Jan 7, 2023

Update GTs for 2023 #40445

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mask old b-taggers and keep only DNN taggers with Era modifier for run3. #39808

Mask old b-taggers and keep only DNN taggers with Era modifier for run3. #39808

yuanchao commented Oct 21, 2022 •

edited

cmsbuild commented Oct 21, 2022

cmsbuild commented Oct 21, 2022

mandrenguyen commented Oct 21, 2022

cmsbuild commented Oct 21, 2022

AlexDeMoor commented Oct 21, 2022

yuanchao commented Oct 24, 2022

yuanchao commented Oct 24, 2022

tvami commented Oct 27, 2022

cmsbuild commented Oct 27, 2022

AlexDeMoor commented Oct 27, 2022

soureek commented Nov 3, 2022

tvami commented Nov 11, 2022

cmsbuild commented Nov 11, 2022

cmsbuild commented Nov 18, 2022

tvami commented Nov 18, 2022

AlexDeMoor commented Nov 22, 2022

clacaputo commented Dec 2, 2022

cmsbuild commented Dec 2, 2022

mandrenguyen commented Dec 8, 2022

tvami commented Dec 8, 2022

mandrenguyen commented Dec 8, 2022

cmsbuild commented Dec 8, 2022

rappoccio commented Dec 9, 2022

Mask old b-taggers and keep only DNN taggers with Era modifier for run3. #39808

Mask old b-taggers and keep only DNN taggers with Era modifier for run3. #39808

Conversation

yuanchao commented Oct 21, 2022 • edited

PR description:

PR validation:

cmsbuild commented Oct 21, 2022

cmsbuild commented Oct 21, 2022

mandrenguyen commented Oct 21, 2022

cmsbuild commented Oct 21, 2022

RelVals

AlexDeMoor commented Oct 21, 2022

yuanchao commented Oct 24, 2022

yuanchao commented Oct 24, 2022

tvami commented Oct 27, 2022

cmsbuild commented Oct 27, 2022

AlexDeMoor commented Oct 27, 2022

soureek commented Nov 3, 2022

tvami commented Nov 11, 2022

cmsbuild commented Nov 11, 2022

RelVals

cmsbuild commented Nov 18, 2022

Comparison Summary

tvami commented Nov 18, 2022

AlexDeMoor commented Nov 22, 2022

clacaputo commented Dec 2, 2022

cmsbuild commented Dec 2, 2022

Comparison Summary

mandrenguyen commented Dec 8, 2022

tvami commented Dec 8, 2022

mandrenguyen commented Dec 8, 2022

cmsbuild commented Dec 8, 2022

rappoccio commented Dec 9, 2022

yuanchao commented Oct 21, 2022 •

edited