Enable tau validation for phase 2 #27120

steggema · 2019-06-05T14:56:18Z

PR description:

This PR enables tau validation for phase 2 and re-adds the RecoTauV directory in the DQM output. It also includes a more optimal setting of the binning for the summary plots in RecoTauV subdirectories. It required changing a few names of duplicated modules for the auto-validation to work since either the wrong one ended up being added to the process or the wrong one was changed by an era modifier.

It may interfere with https://github.com/cms-sw/cmssw/pull/27084/files but merging by hand should be trivial.

PR validation:

Checked that the RecoTauV folder is added with appropriate content for the relevant matrix workflows 27434.0, 29434.0, 20434.0, 20034.0

if this PR is a backport please specify the original PR:

Adding @rmanzoni @roger-wolf @mbluj @aspiezia

…mbers of working points change

…on is not semi-randomly picked by which central import comes first and (b) era modifiers don't act in unforeseen places

cmsbuild · 2019-06-05T14:56:52Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-06-05T15:02:17Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27120/10236

This PR adds an extra 28KB to repository

cmsbuild · 2019-06-05T15:02:57Z

A new Pull Request was created by @steggema (Jan Steggemann) for master.

It involves the following packages:

DQMOffline/Configuration
FastSimulation/Configuration
Validation/Configuration
Validation/RecoTau

@ssekmen, @lveldere, @kmaeshima, @schneiml, @andrius-k, @mdhildreth, @cmsbuild, @jfernan2, @fioriNTU, @civanch can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @matt-komm, @threus, @rovere, @rociovilar this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

jfernan2 · 2019-06-05T15:18:53Z

Validation/RecoTau/src/TauTagValidation.cc

@@ -129,7 +129,7 @@ void TauTagValidation::bookHistograms(DQMStore::IBooker& ibooker,
  ibooker.setCurrentFolder("RecoTauV/" + TauProducer_ + extensionName_ + "_Summary");
  hinfo summaryHinfo = (histoSettings_.exists("summary"))
                           ? hinfo(histoSettings_.getParameter<edm::ParameterSet>("summary"))
-                           : hinfo(21, -0.5, 20.5);
+                           : hinfo(discriminators_.size(), -0.5, discriminators_.size() - 0.5);


@steggema is this change creating a non null number of bins histogram in all cases?

For all regular cases yes, but it may be a good idea to add a check here.

Please do so, we are having issues while merging such histograms: #26727

This was addressed by af33e54.
Also merged with the latest master to get in sync with the meanwhile merged E/Gamma PR mentioned in the PR description).

jfernan2 · 2019-06-05T15:21:32Z

Validation/RecoTau/python/dataTypes/ValidateTausOnZTT_cff.py

@@ -5,9 +5,11 @@
 from PhysicsTools.HepMCCandAlgos.genParticles_cfi import *
 import PhysicsTools.PatAlgos.tools.helpers as helpers

+tauGenJetsForVal = tauGenJets.clone()


@steggema I don't understand why this clone is needed since no modifications are done afterwards

The point is that a MiniAOD era modifier changes process.tauGenJets, so it needs to have a different name and can't be the same object (i.e. if you write tauGenJetsForVal = tauGenJets it will just be a pointer). Maybe there's a better solution (... import as ... didn't work either).

I see thanks

cmsbuild · 2019-06-06T08:14:17Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-06-06T08:21:54Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27120/10247

This PR adds an extra 28KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-checks:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27120/10247/code-checks.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27120/10247/code-checks.patch | patch -p1
You can also run scram build code-checks to apply code checks directly

jfernan2 · 2019-06-06T15:23:12Z

please test

cmsbuild · 2019-06-06T15:23:41Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/800/console Started: 2019/06/06 17:24

cmsbuild · 2019-06-06T20:31:33Z

+1
Tested at: 29a76e9
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-dc52c5/800/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

Running code-format for daq-l1-reconstruction #27110 @cms-sw: Running code-format for daq-l1-reconstruction
Running code-format for analysis-reconstruction #27114 @cms-sw: Running code-format for analysis-reconstruction
DD4hep: Augment tested features and enable runing the tests without disabling biglibs #26998 @ianna: DD4hep: Augment tested features and enable runing the tests without disabling biglibs
Running code-format for reconstruction #27080 @cms-sw: Running code-format for reconstruction
Running code-format for reconstruction #27078 @cms-sw: Running code-format for reconstruction
Running code-format for reconstruction #27076 @cms-sw: Running code-format for reconstruction
Running code-format for reconstruction #27075 @cms-sw: Running code-format for reconstruction
Running code-format for reconstruction #27074 @cms-sw: Running code-format for reconstruction

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-dc52c5/800/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-dc52c5/800/git-merge-result

cmsbuild · 2019-06-06T20:31:37Z

Comparison job queued.

cmsbuild · 2019-06-06T23:42:54Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-dc52c5/800/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 16 differences found in the comparisons
DQMHistoTests: Total files compared: 33
DQMHistoTests: Total histograms compared: 3215458
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 102
DQMHistoTests: Total successes: 3215021
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 5766.462 KiB( 32 files compared)
DQMHistoSizes: changed ( 136.731,... ): -0.188 KiB RecoTauV/hpsPFTauProducerRealMuonsData_Summary
DQMHistoSizes: changed ( 136.731,... ): -0.152 KiB RecoTauV/hpsPFTauProducerRealElectronsData_Summary
DQMHistoSizes: changed ( 10042.0,... ): -0.188 KiB RecoTauV/hpsPFTauProducerZMM_Summary
DQMHistoSizes: changed ( 10042.0,... ): -0.152 KiB RecoTauV/hpsPFTauProducerZEE_Summary
DQMHistoSizes: changed ( 10042.0,... ): 0.082 KiB RecoTauV/hpsPFTauProducerZTT_Summary
DQMHistoSizes: changed ( 21234.0,... ): 17.333 KiB RecoTauV/hpsPFTauProducerZTT_hpsPFTauDiscriminationByMediumCombinedIsolationDBSumPtCorr3HitsdR03
DQMHistoSizes: changed ( 21234.0,... ): 17.333 KiB RecoTauV/hpsPFTauProducerQCD_hpsPFTauDiscriminationByMediumCombinedIsolationDBSumPtCorr3HitsdR03
DQMHistoSizes: changed ( 21234.0,... ): 17.253 KiB RecoTauV/hpsPFTauProducerQCD_hpsPFTauDiscriminationByLooseCombinedIsolationDBSumPtCorr3HitsdR03
DQMHistoSizes: changed ( 21234.0,... ): 17.253 KiB RecoTauV/hpsPFTauProducerQCD_hpsPFTauDiscriminationByTightCombinedIsolationDBSumPtCorr3HitsdR03
DQMHistoSizes: changed ( 21234.0,... ): 17.253 KiB RecoTauV/hpsPFTauProducerZTT_hpsPFTauDiscriminationByLooseCombinedIsolationDBSumPtCorr3HitsdR03
DQMHistoSizes: changed ( 21234.0 ): ...
Checked 137 log files, 14 edm output root files, 33 DQM output files

jfernan2 · 2019-06-07T06:55:13Z

@steggema Before approving, can you please do a last check that the 4069 histograms added to Phase2 wf are what you expect? E.g. in http://tinyurl.com/y4yc6ffk

steggema · 2019-06-07T07:53:47Z

@steggema Before approving, can you please do a last check that the 4069 histograms added to Phase2 wf are what you expect? E.g. in http://tinyurl.com/y4yc6ffk

@jfernan2 Yes, they're as expected for MC workflows.

ssekmen · 2019-06-07T07:56:04Z

+1

jfernan2 · 2019-06-07T10:00:16Z

+1

civanch · 2019-06-10T13:00:20Z

+1

cmsbuild · 2019-06-10T13:00:49Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

fabiocos · 2019-06-12T08:38:29Z

+1

the memory addition to the total is relatively small

< Total bytes: 675105.24 KiB
---
> Total bytes: 673661.26 KiB

and dispersed in a large number of small subfolders (not visible with the standard threshold of 512 KiB).

steggema added 4 commits June 5, 2019 13:16

Add taus to phase 2 validation

ccb41fe

Avoid overflows in summary histogram in case discriminators and or nu…

014e678

…mbers of working points change

Use unique CMSSW module names such that (a) sequence in auto validati…

f2e79c7

…on is not semi-randomly picked by which central import comes first and (b) era modifiers don't act in unforeseen places

Adapt name of tau pre-validation sequence also in FastSim DQM

6c54299

cmsbuild added this to the CMSSW_11_0_X milestone Jun 5, 2019

cmsbuild added code-checks-pending comparison-pending dqm-pending fastsim-pending orp-pending pending-signatures simulation-pending tests-pending labels Jun 5, 2019

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 5, 2019

jfernan2 reviewed Jun 5, 2019

View reviewed changes

Protect against having histogram with no bins

751242d

cmsbuild added code-checks-pending and removed code-checks-approved labels Jun 6, 2019

cmsbuild added code-checks-rejected and removed code-checks-pending labels Jun 6, 2019

Code-check (better test for empty vector)

af33e54

cmsbuild added code-checks-pending and removed code-checks-rejected labels Jun 6, 2019

cmsbuild added tests-started and removed tests-pending labels Jun 6, 2019

cmsbuild added tests-approved and removed tests-started labels Jun 6, 2019

cmsbuild added comparison-available and removed comparison-pending labels Jun 6, 2019

cmsbuild added fastsim-approved and removed fastsim-pending labels Jun 7, 2019

cmsbuild added dqm-approved and removed dqm-pending labels Jun 7, 2019

cmsbuild added fully-signed simulation-approved and removed pending-signatures simulation-pending labels Jun 10, 2019

cmsbuild added orp-approved and removed orp-pending labels Jun 12, 2019

cmsbuild merged commit 786d2b9 into cms-sw:master Jun 12, 2019

cmsbuild mentioned this pull request Jun 12, 2019

Fix python3 interface problem in RecoTauTag/RecoTau #27189

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable tau validation for phase 2 #27120

Enable tau validation for phase 2 #27120

steggema commented Jun 5, 2019

cmsbuild commented Jun 5, 2019

cmsbuild commented Jun 5, 2019

cmsbuild commented Jun 5, 2019

jfernan2 Jun 5, 2019

steggema Jun 5, 2019

jfernan2 Jun 5, 2019

steggema Jun 6, 2019

jfernan2 Jun 5, 2019

steggema Jun 5, 2019

jfernan2 Jun 5, 2019

cmsbuild commented Jun 6, 2019

cmsbuild commented Jun 6, 2019

jfernan2 commented Jun 6, 2019

cmsbuild commented Jun 6, 2019 •

edited

cmsbuild commented Jun 6, 2019

cmsbuild commented Jun 6, 2019

cmsbuild commented Jun 6, 2019

jfernan2 commented Jun 7, 2019

steggema commented Jun 7, 2019

ssekmen commented Jun 7, 2019

jfernan2 commented Jun 7, 2019

civanch commented Jun 10, 2019

cmsbuild commented Jun 10, 2019

fabiocos commented Jun 12, 2019

Enable tau validation for phase 2 #27120

Enable tau validation for phase 2 #27120

Conversation

steggema commented Jun 5, 2019

PR description:

PR validation:

if this PR is a backport please specify the original PR:

cmsbuild commented Jun 5, 2019

cmsbuild commented Jun 5, 2019

cmsbuild commented Jun 5, 2019

jfernan2 Jun 5, 2019

Choose a reason for hiding this comment

steggema Jun 5, 2019

Choose a reason for hiding this comment

jfernan2 Jun 5, 2019

Choose a reason for hiding this comment

steggema Jun 6, 2019

Choose a reason for hiding this comment

jfernan2 Jun 5, 2019

Choose a reason for hiding this comment

steggema Jun 5, 2019

Choose a reason for hiding this comment

jfernan2 Jun 5, 2019

Choose a reason for hiding this comment

cmsbuild commented Jun 6, 2019

cmsbuild commented Jun 6, 2019

jfernan2 commented Jun 6, 2019

cmsbuild commented Jun 6, 2019 • edited

cmsbuild commented Jun 6, 2019

cmsbuild commented Jun 6, 2019

cmsbuild commented Jun 6, 2019

jfernan2 commented Jun 7, 2019

steggema commented Jun 7, 2019

ssekmen commented Jun 7, 2019

jfernan2 commented Jun 7, 2019

civanch commented Jun 10, 2019

cmsbuild commented Jun 10, 2019

fabiocos commented Jun 12, 2019

cmsbuild commented Jun 6, 2019 •

edited