Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix cluster-to-TrackingParticle associations for duplicate entries #19637

Merged
merged 1 commit into from
Aug 9, 2017
Merged

Fix cluster-to-TrackingParticle associations for duplicate entries #19637

merged 1 commit into from
Aug 9, 2017

Conversation

makortel
Copy link
Contributor

I noticed the ClusterTPAssociation can contain multiple associations from a cluster to a single TrackingParticle. This can happen despite of ClusterTPAssociationProducer already having already a logic for adding only one entry for each cluster-to-SimTrack association. But electron TrackingParticles can consist of multiple SimTracks, and therefore the existing logic is not enough. I decided to enhance the sort comparison function and use std::unique at the end as that's likely faster than doing any additional existence checks before the insertion, especially given that it does not happen very often.

While e.g. the track-to-TrackingParticle association (i.e. QuickTrackAssociatorByHits) is not affected, some other analyses may be affected because of the duplicated entries. E.g. here is one affected piece from standard workflows (FYI @smoortga)
https://github.com/cms-sw/cmssw/blob/master/Validation/RecoB/plugins/BDHadronTrackMonitoringAnalyzer.cc#L218-L246
(this PR provides a bug fix and in all cases the number of clusters / TrackingParticle should decrease)

Tested in CMSSW_9_1_0_pre3, expecting changes in the BDHadron monitoring.

@rovere @VinInn

@makortel
Remove duplicate cluster->TP associations
734f8c6 
The duplicates cause problems in certain kind of bookkeeping.
@cmsbuild cmsbuild added this to the CMSSW_9_3_X milestone Jul 10, 2017
@cmsbuild
Copy link
Contributor

A new Pull Request was created by @makortel (Matti Kortelainen) for master.

It involves the following packages:

SimTracker/TrackerHitAssociation

@cmsbuild, @civanch, @mdhildreth, @davidlange6 can you please review it and eventually sign? Thanks.
@sdevissc, @GiacomoSguazzoni, @rovere, @VinInn, @wmtford, @ebrondol, @threus, @dgulhan this is something you requested to watch as well.
@davidlange6 you are the release manager for this.

cms-bot commands are listed here

@makortel
Copy link
Contributor Author

@cmsbuild, please tesst

@makortel
Copy link
Contributor Author

@cmsbuild, please test

(hopefully without the typo this time...)

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 10, 2017 •
edited
Loading

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/21304/console Started: 2017/07/10 14:21

@makortel
Copy link
Contributor Author

type bug-fix

@cmsbuild
Copy link
Contributor

-1

Tested at: 734f8c6

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
0619998
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19637/21304/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19637/21304/git-merge-result

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19637/21304/summary.html

I found follow errors while testing this PR

Failed tests: AddOn

  • AddOn:

I found errors in the following addon tests:

cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_GRun --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2017 --magField 38T_PostLS1 --fileout file:RelVal_Raw_GRun_MC.root : FAILED - time: date Mon Jul 10 15:26:18 2017-date Mon Jul 10 15:24:41 2017 s - exit: 21248
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02480/slc6_amd64_gcc530/cms/cmssw-patch/CMSSW_9_3_X_2017-07-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_GRun.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Mon Jul 10 15:26:18 2017-date Mon Jul 10 15:24:41 2017 s - exit: 21504
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2017 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_GRun_MC.root --fileout file:RelVal_Raw_GRun_MC_HLT_RECO.root : FAILED - time: date Mon Jul 10 15:26:18 2017-date Mon Jul 10 15:24:41 2017 s - exit: 21248
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_25ns --magField 38T_PostLS1 : FAILED - time: date Mon Jul 10 15:25:42 2017-date Mon Jul 10 15:24:45 2017 s - exit: 21248
cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_PRef --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2017 --magField 38T_PostLS1 --fileout file:RelVal_Raw_PRef_MC.root : FAILED - time: date Mon Jul 10 15:25:58 2017-date Mon Jul 10 15:24:49 2017 s - exit: 21248
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02480/slc6_amd64_gcc530/cms/cmssw-patch/CMSSW_9_3_X_2017-07-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PRef.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Mon Jul 10 15:25:58 2017-date Mon Jul 10 15:24:49 2017 s - exit: 21504
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2017 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PRef_MC.root --fileout file:RelVal_Raw_PRef_MC_HLT_RECO.root : FAILED - time: date Mon Jul 10 15:25:58 2017-date Mon Jul 10 15:24:49 2017 s - exit: 21248
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_2016 --magField 38T_PostLS1 : FAILED - time: date Mon Jul 10 15:26:10 2017-date Mon Jul 10 15:24:51 2017 s - exit: 21248
cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_GRun2016 --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2016 --magField 38T_PostLS1 --fileout file:RelVal_Raw_GRun2016_MC.root : FAILED - time: date Mon Jul 10 15:26:44 2017-date Mon Jul 10 15:24:53 2017 s - exit: 21248
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02480/slc6_amd64_gcc530/cms/cmssw-patch/CMSSW_9_3_X_2017-07-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_GRun2016.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Mon Jul 10 15:26:44 2017-date Mon Jul 10 15:24:53 2017 s - exit: 21504
cmsDriver.py RelVal -s HLT:GRun2016,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_GRun2016 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_GRun2016_MC.root --fileout file:RelVal_Raw_GRun2016_MC_HLT_RECO.root : FAILED - time: date Mon Jul 10 15:26:44 2017-date Mon Jul 10 15:24:53 2017 s - exit: 21248
cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=HeavyIons -n 10 --conditions auto:run2_mc_HIon --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2016,Run2_HI --magField 38T_PostLS1 --fileout file:RelVal_Raw_HIon_MC.root : FAILED - time: date Mon Jul 10 15:26:06 2017-date Mon Jul 10 15:25:03 2017 s - exit: 21248
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02480/slc6_amd64_gcc530/cms/cmssw-patch/CMSSW_9_3_X_2017-07-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_HIon.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Mon Jul 10 15:26:06 2017-date Mon Jul 10 15:25:03 2017 s - exit: 21504
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --mc --scenario=HeavyIons -n 10 --conditions auto:run2_mc_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016,Run2_HI --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_HIon_MC.root --fileout file:RelVal_Raw_HIon_MC_HLT_RECO.root : FAILED - time: date Mon Jul 10 15:26:06 2017-date Mon Jul 10 15:25:03 2017 s - exit: 21248
cmsDriver.py TTbar_Tauola_13TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run2_mc_PIon --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2017 --magField 38T_PostLS1 --fileout file:RelVal_Raw_PIon_MC.root : FAILED - time: date Mon Jul 10 15:26:46 2017-date Mon Jul 10 15:25:26 2017 s - exit: 21248
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02480/slc6_amd64_gcc530/cms/cmssw-patch/CMSSW_9_3_X_2017-07-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PIon.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Mon Jul 10 15:26:46 2017-date Mon Jul 10 15:25:26 2017 s - exit: 21504
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2017 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PIon_MC.root --fileout file:RelVal_Raw_PIon_MC_HLT_RECO.root : FAILED - time: date Mon Jul 10 15:26:46 2017-date Mon Jul 10 15:25:26 2017 s - exit: 21248

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
0619998
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19637/21304/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19637/21304/git-merge-result

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19637/21304/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 23
  • DQMHistoTests: Total histograms compared: 1884334
  • DQMHistoTests: Total failures: 117
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 1884051
  • DQMHistoTests: Total skipped: 166
  • DQMHistoTests: Total Missing objects: 0
  • Checked 93 log files, 14 edm output root files, 23 DQM output files

@tankit tankit mentioned this pull request Jul 12, 2017
Closed
@makortel
Copy link
Contributor Author

makortel commented Aug 7, 2017

@cmsbuild, please test

Let's try again

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 7, 2017 •
edited
Loading

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/22125/console Started: 2017/08/07 15:30

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 7, 2017

+1
Tested at: 734f8c6
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19637/22125/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
20cb694
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19637/22125/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19637/22125/git-merge-result

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 7, 2017

Comparison job queued.

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 7, 2017

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19637/22125/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 25
  • DQMHistoTests: Total histograms compared: 2646495
  • DQMHistoTests: Total failures: 195
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2646119
  • DQMHistoTests: Total skipped: 181
  • DQMHistoTests: Total Missing objects: 0
  • Checked 102 log files, 14 edm output root files, 25 DQM output files

@civanch
Copy link
Contributor

civanch commented Aug 8, 2017

+1

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 8, 2017

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @smuzaffar (and backports should be raised in the release meeting by the corresponding L2)

@davidlange6
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit 1464f2f into cms-sw:master Aug 9, 2017
@makortel makortel deleted the clusterTPAssociationSort branch February 12, 2018 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug-fix comparison-available fully-signed orp-approved simulation-approved tests-approved
Projects
None yet
Milestone
CMSSW_9_3_X
Development

Successfully merging this pull request may close these issues.
4 participants
@makortel @cmsbuild @civanch @davidlange6