Sort pixel tracks in the SoA converter (12_4_X) #38066

silviodonato · 2022-05-24T16:29:51Z

Backport of #38065 to 12_4_X

PR description:

This PR helps to reduce the CPU-GPU differences. Testing on 10k of events of 900 GeV collisions, the hltAK4PFJets with difference in pt greater than 0.35 GeV passed from 5 to 2.

Moreover, having the pixel tracks sorted by pT helps the CPU-GPU comparison.

PR validation:

I checked on 10k of events that this PR does not change the CPU reconstruction, while it changes 3 jets reconstructed using GPU, reducing the differences from CPU.

root [2] Events->Scan("EventAuxiliary.event():EventAuxiliary.run():recoPFJets_hltAK4PFJets__CPU3.obj.pt():recoPFJets_hltAK4PFJets__GPU3.obj.pt():recoPFJets_hltAK4PFJets__CPU4.obj.pt():recoPFJets_hltAK4PFJets__GPU4.obj.pt()","abs(recoPFJets_hltAK4PFJets__CPU4.obj.pt()-recoPFJets_hltAK4PFJets__GPU4.obj.pt())>0.35 || abs(recoPFJets_hltAK4PFJets__CPU3.obj.pt()-recoPFJets_hltAK4PFJets__GPU3.obj.pt())>0.35")
***********************************************************************************************
*    Row   * Instance * EventAuxi * EventAuxi * recoPFJet * recoPFJet * recoPFJet * recoPFJet *
***********************************************************************************************
*     2787 *        1 * 249032616 *    346512 * 1.9907082 * 1.6183525 * 1.9907082 * 1.6183525 *
*     4010 *        2 * 251712216 *    346512 * 2.0095655 * 2.0095655 * 2.0095655 * 2.6015811 *
*     4010 *        3 * 251712216 *    346512 * 1.6040698 * 1.6040675 * 1.6040698 * 2.0095655 *
*     7289 *        2 * 260018739 *    346512 * 1.7360442 * 2.2783870 * 1.7360442 * 2.2783870 *
*     7962 *        1 * 261754240 *    346512 * 0.8782627 * 0.8782627 * 0.8782627 * 1.2651753 *
***********************************************************************************************

CPU3 and GPU3 are after the PR and CPU4 and GPU4 before the PR

Backport:

I will open soon the backport PRs to use this PR in the HLT at P5

cc @cms-sw/hlt-l2 @fwyzard @VinInn

cmsbuild · 2022-05-24T16:30:15Z

A new Pull Request was created by @silviodonato (Silvio Donato) for CMSSW_12_4_X.

It involves the following packages:

RecoPixelVertexing/PixelTrackFitting (reconstruction)

@jpata, @cmsbuild, @clacaputo, @slava77 can you please review it and eventually sign? Thanks.
@felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @mmusich, @mtosi, @dgulhan this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

Backported from Sort pixel tracks in the SoA converter #38065

cmsbuild · 2022-05-24T19:21:02Z

Pull request #38066 was updated. @jpata, @cmsbuild, @clacaputo, @slava77 can you please check and sign again.

cmsbuild · 2022-05-25T10:55:50Z

Pull request #38066 was updated. @jpata, @cmsbuild, @clacaputo, @slava77 can you please check and sign again.

jpata · 2022-05-25T13:46:14Z

@cmsbuild please test

mmusich · 2022-05-25T14:01:41Z

hi @jpata I think that the discussion about filtering on quality of the track in the master PR hasn't converged.
Also the tests here need to be configured to run GPU wf as well.
Can you cancel the test to avoid wasting resources? Thank you.

jpata · 2022-05-25T14:07:00Z

@cmsbuild please cancel

cmsbuild · 2022-05-25T17:28:49Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9b64fe/24990/summary.html
COMMIT: 3eeb421
CMSSW: CMSSW_12_4_X_2022-05-25-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38066/24990/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3672464
DQMHistoTests: Total failures: 1138
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3671304
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 208 log files, 45 edm output root files, 50 DQM output files
TriggerResults: found differences in 2 / 49 workflows

cmsbuild · 2022-05-30T16:16:45Z

Pull request #38066 was updated. @jpata, @cmsbuild, @clacaputo, @slava77 can you please check and sign again.

fwyzard · 2022-05-30T16:46:46Z

please test

cmsbuild · 2022-05-30T21:02:05Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9b64fe/25081/summary.html
COMMIT: 7efac99
CMSSW: CMSSW_12_4_X_2022-05-30-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38066/25081/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3668385
DQMHistoTests: Total failures: 1137
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3667225
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 49 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 208 log files, 45 edm output root files, 50 DQM output files
TriggerResults: found differences in 2 / 49 workflows

mmusich · 2022-06-01T12:48:47Z

type tracking

missirol · 2022-06-09T06:10:21Z

urgent

To signal that TSG targets 12_4_0 for this PR.

missirol · 2022-06-09T06:11:15Z

test parameters:

enable_tests = gpu
workflows_gpu = 11634.503, 11634.506, 11634.583, 11634.587
workflow = 11634.501
relvals_opt = -w standard,highstats,pileup,generator,extendedgen,production,upgrade,cleanedupgrade,ged

missirol · 2022-06-09T06:12:29Z

please test

Re-running the tests as in the original PR, since outputs of previous tests expired.

cmsbuild · 2022-06-09T10:30:37Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9b64fe/25380/summary.html
COMMIT: 7efac99
CMSSW: CMSSW_12_4_X_2022-06-08-2300/el8_amd64_gcc10
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38066/25380/install.sh to create a dev area with all the needed externals and cmssw changes.

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 76363
DQMHistoTests: Total failures: 84
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 76279
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 24 log files, 18 edm output root files, 7 DQM output files
TriggerResults: found differences in 6 / 6 workflows

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3687504
DQMHistoTests: Total failures: 1150
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3686332
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
Checked 212 log files, 48 edm output root files, 51 DQM output files
TriggerResults: found differences in 3 / 50 workflows

jpata · 2022-06-09T10:40:24Z

+reconstruction

no relevant differences in reco (some in HLT)
addresses CPU-GPU differences at HLT
backport of Sort pixel tracks in the SoA converter #38065

cmsbuild · 2022-06-09T10:40:48Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_4_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_5_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

missirol · 2022-06-10T16:30:59Z

backport of #38065

qliphy · 2022-06-11T00:52:36Z

+1

cmsbuild added this to the CMSSW_12_4_X milestone May 24, 2022

cmsbuild added orp-pending pending-signatures reconstruction-pending tests-pending labels May 24, 2022

silviodonato mentioned this pull request May 24, 2022

Sort pixel tracks in the SoA converter #38065

Merged

cmsbuild added tests-started and removed tests-pending labels May 25, 2022

cmsbuild added tests-approved and removed tests-started labels May 25, 2022

sort pixel tracks from SoA by pt

7efac99

silviodonato force-pushed the sortPixelTracks branch from 3eeb421 to 7efac99 Compare May 30, 2022 16:16

cmsbuild added tests-pending and removed tests-approved labels May 30, 2022

cmsbuild added tests-started and removed tests-pending labels May 30, 2022

cmsbuild added tests-approved and removed tests-started labels May 30, 2022

cmsbuild added the tracking label Jun 1, 2022

cmsbuild added the urgent label Jun 9, 2022

cmsbuild added tests-started and removed tests-approved labels Jun 9, 2022

cmsbuild added tests-approved and removed tests-started labels Jun 9, 2022

cmsbuild added fully-signed reconstruction-approved and removed reconstruction-pending pending-signatures labels Jun 9, 2022

cmsbuild added the backport-ok label Jun 10, 2022

cmsbuild added orp-approved and removed orp-pending labels Jun 11, 2022

cmsbuild merged commit 667de6e into cms-sw:CMSSW_12_4_X Jun 11, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sort pixel tracks in the SoA converter (12_4_X) #38066

Sort pixel tracks in the SoA converter (12_4_X) #38066

silviodonato commented May 24, 2022

cmsbuild commented May 24, 2022 •

edited

cmsbuild commented May 24, 2022

cmsbuild commented May 25, 2022

jpata commented May 25, 2022

mmusich commented May 25, 2022

jpata commented May 25, 2022

cmsbuild commented May 25, 2022

cmsbuild commented May 30, 2022

fwyzard commented May 30, 2022

cmsbuild commented May 30, 2022

mmusich commented Jun 1, 2022

missirol commented Jun 9, 2022

missirol commented Jun 9, 2022

missirol commented Jun 9, 2022

cmsbuild commented Jun 9, 2022

jpata commented Jun 9, 2022

cmsbuild commented Jun 9, 2022

missirol commented Jun 10, 2022

qliphy commented Jun 11, 2022

Sort pixel tracks in the SoA converter (12_4_X) #38066

Sort pixel tracks in the SoA converter (12_4_X) #38066

Conversation

silviodonato commented May 24, 2022

PR description:

PR validation:

Backport:

cmsbuild commented May 24, 2022 • edited

cmsbuild commented May 24, 2022

cmsbuild commented May 25, 2022

jpata commented May 25, 2022

mmusich commented May 25, 2022

jpata commented May 25, 2022

cmsbuild commented May 25, 2022

Comparison Summary

cmsbuild commented May 30, 2022

fwyzard commented May 30, 2022

cmsbuild commented May 30, 2022

Comparison Summary

mmusich commented Jun 1, 2022

missirol commented Jun 9, 2022

missirol commented Jun 9, 2022

missirol commented Jun 9, 2022

cmsbuild commented Jun 9, 2022

GPU Comparison Summary

Comparison Summary

jpata commented Jun 9, 2022

cmsbuild commented Jun 9, 2022

missirol commented Jun 10, 2022

qliphy commented Jun 11, 2022

cmsbuild commented May 24, 2022 •

edited