Renewed PF validation #27048

hatakeyamak · 2019-05-31T22:02:26Z

PR description:

Renewed PF validation version 1. We would like to include this in the regular validation/DQM sequence, but we also intend to use them to run on the fly to test planned PF optimization/development. The currently added histograms are for evaluation jet response, resolution, and offset distributions.

PR validation:

The example plots look like:
2018 MC: http://hep.baylor.edu/hatake/PFval/val_10_6_0_test1/
Phase 2: http://hep.baylor.edu/hatake/PFval/val_10_6_0_phase2/
Also they were tested with various matrix tests:
10024.0
10824.0
136.888
2018.1
29034.0

if this PR is a backport please specify the original PR:

Not applicable.

@jpata @juska @bendavid

…t plots

Added the "voms-proxy-init -voms cms" line for accessing files over xrootd. I already added this once to the readme, but it seems to have disappeared.

…xis title option, updated good npv def

Added a .PHONY command for re-running plots. Without it the makefile gives "already up to date" when trying to re-run with changed input file content.

…ecks in offsetStack

… for gaussian fit as it really jumps for now. now show both rms version and gaussian fit version

Add comment on running 'make plots'

….0] and [4.0,5.0]. Hide also fitting pop-up windows.

cmsbuild · 2019-06-26T15:57:05Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27048/10629

This PR adds an extra 688KB to repository
Found files with invalid states:
- Validation/RecoParticleFlow/test/Jet.py:
  - Added: 0409088
  - Deleted: 1abf33f
- Validation/RecoParticleFlow/crab/step3_2018.py:
  - Added: 7dfcc17
  - Deleted: 5c1a674
- Validation/RecoParticleFlow/tmp/das_cache/QCD_PU.txt:
  - Added: d69d7ab
  - Deleted: 2c82951
- Validation/RecoParticleFlow/crab/step3_dump.py:
  - Added: bec5db9
  - Modified: 5c1a674
  - Deleted: d69d7ab
- Validation/RecoParticleFlow/test/addResponseAndResolution.py:
  - Added: 4dea4a5
  - Modified: fc6140a, 7b3724a, b1a7fee, 84598f9
  - Deleted: 1abf33f
- Validation/RecoParticleFlow/test/addResponse.py:
  - Added: 0409088
  - Modified: fc6140a
  - Deleted: 1abf33f
- Validation/RecoParticleFlow/tmp/das_cache/ZMM.txt:
  - Added: d69d7ab
  - Deleted: 2c82951
- Validation/RecoParticleFlow/tmp/das_cache/QCD_noPU.txt:
  - Added: d69d7ab
  - Deleted: 2c82951
- Validation/RecoParticleFlow/test/pfreso.py:
  - Added: 0409088
  - Deleted: 1abf33f
- Validation/RecoParticleFlow/crab/multicrab_phase2.py:
  - Added: d69d7ab
  - Modified: 82010b8, 7b22bee
  - Deleted: 2c82951
- Validation/RecoParticleFlow/tmp/das_cache/QCD_FlatPt_noPU/RelValQCD_FlatPt_15_3000HS_13__CMSSW_10_4_0_pre4-103X_mc2017_realistic_v2-v1__GEN-SIM-DIGI-RAW.txt:
  - Added: 0409088
  - Deleted: 5c1a674
- Validation/RecoParticleFlow/crab/multicrab_2018.py:
  - Added: 7dfcc17
  - Modified: 3924148
  - Deleted: 5c1a674
- Validation/RecoParticleFlow/tmp/das_cache/MinBias/RelValMinBias_13__CMSSW_10_4_0_pre4-103X_mc2017_realistic_v2-v1__GEN-SIM-DIGI-RAW.txt:
  - Added: 0409088
  - Deleted: 5c1a674
- Validation/RecoParticleFlow/test/validor.py:
  - Added: 0409088
  - Deleted: 1abf33f
- Validation/RecoParticleFlow/tmp/das_cache/MinBias.txt:
  - Added: d69d7ab
  - Deleted: 2c82951
- Validation/RecoParticleFlow/tmp/das_cache/QCD_FlatPt_PU25ns/RelValQCD_FlatPt_15_3000HS_13__CMSSW_10_4_0_pre4-PU25ns_103X_mc2017_realistic_v2-v1__GEN-SIM-DIGI-RAW.txt:
  - Added: 0409088
  - Deleted: 5c1a674
- Validation/RecoParticleFlow/tmp/das_cache/NuGun_PU.txt:
  - Added: d69d7ab
  - Deleted: 2c82951
- Validation/RecoParticleFlow/crab/multicrab.py:
  - Added: bec5db9
  - Modified: 7dfcc17, 3924148, 5c1a674, 7b22bee
  - Deleted: 2c82951
- Validation/RecoParticleFlow/plugins/ParticleFlowDQM.cc:
  - Added: 0409088
  - Modified: dd80a0e, 4eae236
  - Deleted: 14c9c85
- Validation/RecoParticleFlow/tmp/das_cache/NuGun_PU25ns/RelValNuGun__CMSSW_10_5_0_pre1-PU25ns_103X_mc2017_realistic_v2_HS-v1__GEN-SIM-DIGI-RAW.txt:
  - Added: 7dfcc17
  - Deleted: 5c1a674
- Validation/RecoParticleFlow/test/helperFunctions.py:
  - Added: 4dea4a5
  - Deleted: 1abf33f
- Validation/RecoParticleFlow/python/DQMForPF_MiniAOD_source_cff.py:
  - Added: 68e8eb7
  - Modified: 387d6f7
  - Deleted: 5d263d6
- Validation/RecoParticleFlow/crab/step3_2018_dump.py:
  - Added: 7dfcc17
  - Deleted: 5c1a674
- Validation/RecoParticleFlow/crab/crab_cfg.py:
  - Added: 0409088
  - Deleted: bec5db9
- Validation/RecoParticleFlow/crab/step3.py:
  - Added: 0409088
  - Modified: bec5db9, 5c1a674
  - Deleted: d69d7ab
- Validation/RecoParticleFlow/tmp/das_cache/ZMM/RelValZMM_13__CMSSW_10_4_0_pre4-103X_mc2017_realistic_v2-v1__GEN-SIM-DIGI-RAW.txt:
  - Added: 0409088
  - Deleted: 5c1a674
- Validation/RecoParticleFlow/test/IO.py:
  - Added: 0409088
  - Deleted: 1abf33f

cmsbuild · 2019-06-26T15:57:39Z

Pull request #27048 was updated. @andrius-k, @kmaeshima, @schneiml, @cmsbuild, @jfernan2, @fioriNTU can you please check and sign again.

fabiocos · 2019-06-27T09:53:11Z

@juska @hatakeyamak now it looks ok, thank you. As far as I am concerned this PR may move forward

fabiocos · 2019-06-27T09:53:33Z

unhold

fabiocos · 2019-06-27T09:53:39Z

please test

cmsbuild · 2019-06-27T09:53:43Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/1204/console Started: 2019/06/27 11:55

cmsbuild · 2019-06-27T11:45:47Z

+1
Tested at: 2c82951
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-728e0b/1204/summary.html

cmsbuild · 2019-06-27T11:45:51Z

Comparison job queued.

cmsbuild · 2019-06-27T13:59:34Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-728e0b/1204/summary.html

Comparison Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 33
DQMHistoTests: Total histograms compared: 3254598
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3254263
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 23583.063 KiB( 32 files compared)
DQMHistoSizes: changed ( 1306.0,... ): 1000.009 KiB ParticleFlow/Offset
DQMHistoSizes: changed ( 1306.0,... ): 72.152 KiB ParticleFlow/JetResponse
DQMHistoSizes: changed ( 1306.0,... ): 0.867 KiB ParticleFlow/GenJets
DQMHistoSizes: changed ( 25.0,... ): 994.806 KiB ParticleFlow/Offset
DQMHistoSizes: changed ( 25.0,... ): 69.504 KiB ParticleFlow/JetResponse
Checked 137 log files, 14 edm output root files, 33 DQM output files

jfernan2 · 2019-06-27T14:24:44Z

+1

cmsbuild · 2019-06-27T14:25:17Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

fabiocos · 2019-06-28T16:37:20Z

+1

jpata and others added 30 commits May 23, 2019 12:41

port

0409088

update README

53bef19

Offset analyzer code for miniAOD and stack plotter

74fb1c5

check number of elements in muHandle (noPU MC files only have 1)

1c83430

save genjet pt spectra

dd80a0e

added offset plotting to compare.py. Need to make html file for offse…

7abf506

…t plots

updated crab run

bec5db9

Update README.md

23efb3a

Added the "voms-proxy-init -voms cms" line for accessing files over xrootd. I already added this once to the readme, but it seems to have disappeared.

offset plotting compare individual components, stack plots, default a…

5bc3fc4

…xis title option, updated good npv def

changed DQM profile error option to correct option

d9eb7af

Update Makefile

b518455

Added a .PHONY command for re-running plots. Without it the makefile gives "already up to date" when trying to re-run with changed input file content.

Add smart gaussian fit based resolution calculation

4dea4a5

Add smart gaussian fit based resolution calculation

254dfc1

fixed legends and offset comparisons in compare.py, and added some ch…

5d92621

…ecks in offsetStack

fix the bin shift error in plots with pt in x. Skip the lowest pt bin…

fc6140a

… for gaussian fit as it really jumps for now. now show both rms version and gaussian fit version

Update README.md

f9132f7

Add comment on running 'make plots'

Extend eta binning for response and resolution to include bins [3.0,4…

7b3724a

….0] and [4.0,5.0]. Hide also fitting pop-up windows.

minor OffsetAnalyzer updates. Created python based stack plot producer

a94e887

updated offset profiles to take total eT per pftype of each event

59f92b1

Update SCRAM_ARCH for SLC7

c925ec6

Polish plot style, merge forward region to one bin: 3<|eta|<5.

b1a7fee

update to run over 2018 samples

7dfcc17

update on multicrab and Makefile

3924148

update

68e8eb7

consolidate sequences

bdef643

cleanup and fix

387d6f7

config update for new postprocessors

c46ab4a

cleanup

5d263d6

config update for new postprocessors

4eae236

adding postprocessors

ed17463

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 26, 2019

cmsbuild removed the hold label Jun 27, 2019

cmsbuild added tests-started and removed tests-pending labels Jun 27, 2019

cmsbuild mentioned this pull request Jun 27, 2019

Ecal DPG - Deploy new software (based on BDT) to recover isolated dead channels… #27175

Merged

cmsbuild added tests-approved and removed tests-started labels Jun 27, 2019

cmsbuild added comparison-available and removed comparison-pending labels Jun 27, 2019

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Jun 27, 2019

cmsbuild added orp-approved and removed orp-pending labels Jun 28, 2019

cmsbuild merged commit acc895c into cms-sw:master Jun 28, 2019

jpata deleted the pfvalidation-10_6_X-master branch October 6, 2021 13:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Renewed PF validation #27048

Renewed PF validation #27048

hatakeyamak commented May 31, 2019 •

edited

cmsbuild commented Jun 26, 2019

cmsbuild commented Jun 26, 2019

fabiocos commented Jun 27, 2019

fabiocos commented Jun 27, 2019

fabiocos commented Jun 27, 2019

cmsbuild commented Jun 27, 2019 •

edited

cmsbuild commented Jun 27, 2019

cmsbuild commented Jun 27, 2019

cmsbuild commented Jun 27, 2019

jfernan2 commented Jun 27, 2019

cmsbuild commented Jun 27, 2019

fabiocos commented Jun 28, 2019

Renewed PF validation #27048

Renewed PF validation #27048

Conversation

hatakeyamak commented May 31, 2019 • edited

PR description:

PR validation:

if this PR is a backport please specify the original PR:

cmsbuild commented Jun 26, 2019

cmsbuild commented Jun 26, 2019

fabiocos commented Jun 27, 2019

fabiocos commented Jun 27, 2019

fabiocos commented Jun 27, 2019

cmsbuild commented Jun 27, 2019 • edited

cmsbuild commented Jun 27, 2019

cmsbuild commented Jun 27, 2019

cmsbuild commented Jun 27, 2019

jfernan2 commented Jun 27, 2019

cmsbuild commented Jun 27, 2019

fabiocos commented Jun 28, 2019

hatakeyamak commented May 31, 2019 •

edited

cmsbuild commented Jun 27, 2019 •

edited