Add GsfTracks to tracking validation #13348

cmsbuild · 2016-02-18T09:59:55Z

This PR adds the monitoring of GsfTracks (electronGsfTracks collection) to tracking validation. The efficiency is defined wrt. TrackingParticles with abs(pdgId) == 11, and fake rate wrt. all TrackingParticles. On the same go, the definition of fake rate for conversions is also changed to be wrt. all TrackingParticles. In order to achieve these nicely, and to again properly support the separate label_tp_effic and label_tp_fake parameters of MultiTrackValidator, the following changes were made

The configuration of TrackingParticleNumberOfLayersProducer is fixed
MTV supports reading TrackingParticleRefVector instead of TrackingParticleCollection
- It would have been easier to just use View<TrackingParticle>, but TrackToTrackingParticleAssociator interface does not support it, and changing it would have had much larger impact. (I'll probably try to do it anyway some day)
TrackingParticleConversionSelector is changed to produce TrackingParticleRefVector (and renamed and turned to edm::global)
TrackValidation_cffis migrated to use TrackingParticleRefVector instead of copies
SimToReco (TP->track) association is created wrt. label_tp_fake TPs instead of label_tp_eff, and label_tp_eff is required to be a subset of label_tp_fake (explicit check is added)
- The SimToReco association is used in efficiency numerator, but also in duplicate rate numerator. For duplicates, the SimToReco must contain associations for TPs from label_tp_fake. For efficiency, these "extra" associations don't matter since the associations are checked only for label_tp_eff.

Tested in CMSSW_8_0_X_2016-02-15-1100 (with and without #13242), expecting changes under Tracking/TrackConversion (for fakes and other track-based histograms), and addition of Tracking/TrackGsf folder.

@rovere @VinInn
Automatically ported from CMSSW_8_0_X #13324 (original by @makortel).

The order makes a difference.

…or instead of TrackingParticleCollection Something like this is necessary to be able to have separate sets of TrackingParticles for efficiency and fake rate plots. It would be easier to just use View<TrackingParticle>, but that would have non-trivial impact on the track-TP associator/association interfaces, so it is left to later time.

…RefSelector

Duplicates are defined as "track associated to TP associated to > 1 tracks". The first association is done with recoToSim, and the second with simToReco. If the efficiency and fake TP sets are different, this leads to (unexpected) effects in duplicate. Naively one would assume that the duplicate definition would be independet of the set of TPs used for efficiencies. The fix is to do the simToReco association wrt. the fake TPs. For efficiency histograms it doesn't matter if the simToReco has entries for TPs not in efficiency TP set.

cmsbuild · 2016-02-18T10:00:52Z

A new Pull Request was created by @cmsbuild for CMSSW_8_1_X.

It involves the following packages:

CommonTools/RecoAlgos
SimGeneral/TrackingAnalysis
Validation/RecoTrack

@civanch, @cvuosalo, @mdhildreth, @cmsbuild, @deguio, @slava77, @vanbesien, @davidlange6 can you please review it and eventually sign? Thanks.
@rappoccio, @jdolen, @battibass, @makortel, @abbiendi, @GiacomoSguazzoni, @jhgoh, @VinInn, @ahinzmann, @istaslis, @rovere, @wmtford, @cerati, @trocino, @dgulhan this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

cvuosalo · 2016-02-18T21:11:22Z

@cmsbuild please test

cmsbuild · 2016-02-18T21:11:51Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/11357/console

cmsbuild · 2016-02-18T23:23:25Z

-1
Tested at: 5eb3f94
When I ran the RelVals I found an error in the following worklfows:
5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13348/11357/summary.html

slava77 · 2016-02-19T05:32:02Z

@cmsbuild please test

cmsbuild · 2016-02-19T05:32:21Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/11374/console

cmsbuild · 2016-02-19T11:34:20Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13348/11374/summary.html

cvuosalo · 2016-02-19T17:03:57Z

+1

For #13324 5eb3f94

Adding GsfTracks to tracking validation. #13324 is the 80X version of this PR, and it has already been approved by Reco.

The code changes are satisfactory, and Jenkins tests against baseline CMSSW_8_1_X_2016-02-18-2300 show only the differences expected for this PR, with examples shown in the #13324 thread.

civanch · 2016-02-19T18:52:44Z

+1

makortel · 2016-02-24T16:02:43Z

ping DQM (@deguio, @vanbesien)

slava77 · 2016-02-26T16:53:53Z

ping DQM (@deguio, @vanbesien)
your signature is needed.
Thank you.

deguio · 2016-03-01T12:03:59Z

+1

cmsbuild · 2016-03-01T12:04:16Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_8_1_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @slava77, @davidlange6, @Degano, @smuzaffar

davidlange6 · 2016-03-02T11:06:50Z

+1

Add GsfTracks to tracking validation

davidlange6 · 2016-03-03T10:23:08Z

@makortel - we have some crashes from this PR. For example:
https://cmssdt.cern.ch/SDT/cgi-bin/buildlogs/slc6_amd64_gcc493/CMSSW_8_1_X_2016-03-02-2300/pyRelValMatrixLogs/run/140.1_QCD_Pt_80_120_13_HI+QCD_Pt_80_120_13_HIINPUT+DIGIHI+RECOHI+HARVESTHI/step3_QCD_Pt_80_120_13_HI+QCD_Pt_80_120_13_HIINPUT+DIGIHI+RECOHI+HARVESTHI.log

An exception of category 'InvalidReference' occurred while
[0] Processing run: 1 lumi: 1 event: 2
[1] Running path 'prevalidation_step'
[2] Calling event method for module MultiTrackValidator/'trackValidatorConversion'
Exception Message:
ClusterTPAssociation has TrackingParticles with ProductID 2:142, but got TrackingParticleRef/Handle/ProductID with ID 0:0. This is typically caused by a configuration error.
----- End Fatal Exception -------------------------------------------------
Another exception was caught while trying to clean up files after the primary fatal exception.

makortel · 2016-03-03T10:33:29Z

@davidlange6 I noticed them too and have already a fix #13570. I'll provide a fix for the failures in HI workflows separately (they are also caused by this PR).

makortel added 9 commits February 16, 2016 09:13

Fix the order of SimHit collections, and customize them for FastSim

cf8ff86

The order makes a difference.

Turn TrackingParticleConversionSelector to edm::global

73b0967

Turn TrackingParticleConversionSelector to TrackingParticleConversion…

9b3a31a

…RefSelector

Migrate TrackValidation_cff to use RefSelectors for TrackingParticles

69d55fe

Use all TrackingParticles for conversion fake rate

0efe675

Add MTV instance for electronGsfTracks

6011c33

Add protection for empty efficiency TrackingParticleRefVector

5eb3f94

cmsbuild added this to the Next CMSSW_8_1_X milestone Feb 18, 2016

cmsbuild added reconstruction-pending simulation-pending dqm-pending pending-signatures tests-pending orp-pending comparison-pending labels Feb 18, 2016

cmsbuild added tests-started and removed tests-pending labels Feb 18, 2016

cmsbuild added tests-rejected and removed tests-started labels Feb 18, 2016

cmsbuild added tests-pending and removed tests-rejected labels Feb 19, 2016

cmsbuild removed the tests-pending label Feb 19, 2016

cmsbuild added tests-approved and removed tests-started labels Feb 19, 2016

cmsbuild added comparison-available and removed comparison-pending labels Feb 19, 2016

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Feb 19, 2016

cmsbuild added simulation-approved and removed simulation-pending labels Feb 19, 2016

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Mar 1, 2016

makortel mentioned this pull request Mar 2, 2016

Update tracking MC validation scripts #13551

Merged

cmsbuild added orp-approved and removed orp-pending labels Mar 2, 2016

cmsbuild added a commit that referenced this pull request Mar 2, 2016

Merge pull request #13348 from makortel/mtvGsfTracks

d59dbdb

Add GsfTracks to tracking validation

cmsbuild merged commit d59dbdb into cms-sw:CMSSW_8_1_X Mar 2, 2016

makortel mentioned this pull request Mar 3, 2016

Fix QuickTrackAssociatorByHits for empty TrackingParticleRefVector #13570

Merged

This was referenced Mar 3, 2016

Fix track validation for heavy ions #13579

Merged

Add GsfTracks to tracking validation #13324

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add GsfTracks to tracking validation #13348

Add GsfTracks to tracking validation #13348

cmsbuild commented Feb 18, 2016

cmsbuild commented Feb 18, 2016

cvuosalo commented Feb 18, 2016

cmsbuild commented Feb 18, 2016

cmsbuild commented Feb 18, 2016

slava77 commented Feb 19, 2016

cmsbuild commented Feb 19, 2016

cmsbuild commented Feb 19, 2016

cvuosalo commented Feb 19, 2016

civanch commented Feb 19, 2016

makortel commented Feb 24, 2016

slava77 commented Feb 26, 2016

deguio commented Mar 1, 2016

cmsbuild commented Mar 1, 2016

davidlange6 commented Mar 2, 2016

davidlange6 commented Mar 3, 2016

makortel commented Mar 3, 2016

Add GsfTracks to tracking validation #13348

Add GsfTracks to tracking validation #13348

Conversation

cmsbuild commented Feb 18, 2016

cmsbuild commented Feb 18, 2016

cvuosalo commented Feb 18, 2016

cmsbuild commented Feb 18, 2016

cmsbuild commented Feb 18, 2016

slava77 commented Feb 19, 2016

cmsbuild commented Feb 19, 2016

cmsbuild commented Feb 19, 2016

cvuosalo commented Feb 19, 2016

civanch commented Feb 19, 2016

makortel commented Feb 24, 2016

slava77 commented Feb 26, 2016

deguio commented Mar 1, 2016

cmsbuild commented Mar 1, 2016

davidlange6 commented Mar 2, 2016

davidlange6 commented Mar 3, 2016

makortel commented Mar 3, 2016