Patatrack: introduce pt-clamping in vertex sorting, add #tracks in track SOA #37281

VinInn · 2022-03-20T10:59:30Z

This PR introduces some long standing improvements in Patatrack that can be useful to POG and PAG using directly SOA products.

Track Pt-clamping is introduced in vertex sorting.
This is supposed to mitigate the effect of spurious tracks with large pt due to miss-measurement, resolution or inaccurate calibration/alignment.

The effect has been tested in Z->tau-tau events that are supposed to be among those who most suffer from signal-vertex miss-identification.
The effect is small if not negligible. see: http://innocent.home.cern.ch/innocent/RelVal/ztt_clamp/plots_selectedPixelVertices/pvtagging.pdf
where we compare clamping at 20, 75 (the default) and 1000 GeV.
This is not unexpected given the resolution of HP quadruplets, those used to build vertices, that reaches ~30% at ~70 GeV.
It may be different with initial calibrations or large inefficiency.

Results for ttbar can be found in
http://innocent.home.cern.ch/innocent/RelVal/gpuMTVclamp/plots_selectedPixelVertices/pvtagging.pdf

The number of tracks is stored in the track SOA.
This should make the use of the SOA more user-friendly.

Default has changed, so expect small regressions.

VinInn · 2022-03-20T11:01:36Z

@cmsbuild, please test

VinInn · 2022-03-20T11:01:43Z

enable gpu

cmsbuild · 2022-03-20T11:08:20Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37281/28908

This PR adds an extra 44KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorKernels.cc modified in PR(s): Use cooperative groups to populate Associations (Histograms) in Pixel Patatrack #35713
- File RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorKernels.cu modified in PR(s): Use cooperative groups to populate Associations (Histograms) in Pixel Patatrack #35713
- File RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorKernelsImpl.h modified in PR(s): Use cooperative groups to populate Associations (Histograms) in Pixel Patatrack #35713

cmsbuild · 2022-03-20T11:08:44Z

A new Pull Request was created by @VinInn (Vincenzo Innocente) for master.

It involves the following packages:

CUDADataFormats/Track (heterogeneous, reconstruction)
RecoPixelVertexing/PixelTrackFitting (reconstruction)
RecoPixelVertexing/PixelTriplets (reconstruction)
RecoPixelVertexing/PixelVertexFinding (reconstruction)

@jpata, @makortel, @fwyzard, @clacaputo, @slava77 can you please review it and eventually sign? Thanks.
@felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @mmusich, @mtosi, @dgulhan this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

VinInn · 2022-03-21T16:44:38Z

what is waiting for?

mmusich · 2022-03-21T18:44:46Z

@smuzaffar tests seem stuck here. Can you verify? Thank you.

smuzaffar · 2022-03-21T19:09:05Z

@mmusich , jenkins was very bus today that is why there was not enough resources to run tests, I see that https://cmssdt.cern.ch/jenkins/job/compare-root-files-short-matrix/49033/console is still running for last 2 hours. Hopefully it will finish soon.

cmsbuild · 2022-03-21T21:33:45Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d7e9ee/23240/summary.html
COMMIT: a375a54
CMSSW: CMSSW_12_4_X_2022-03-20-2300/slc7_amd64_gcc10
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37281/23240/install.sh to create a dev area with all the needed externals and cmssw changes.

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 24 differences found in the comparisons
DQMHistoTests: Total files compared: 4
DQMHistoTests: Total histograms compared: 19811
DQMHistoTests: Total failures: 2095
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 17716
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
Checked 12 log files, 9 edm output root files, 4 DQM output files
TriggerResults: no differences found

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-d7e9ee/39434.911_TTbar_14TeV+2026D88_DD4hep+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3695650
DQMHistoTests: Total failures: 51619
DQMHistoTests: Total nulls: 67
DQMHistoTests: Total successes: 3643942
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 204 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

jpata · 2022-03-22T15:04:04Z

assign tracking
FYI @cms-sw/tracking-pog-l2

mmusich · 2022-03-28T07:41:44Z

assign tracking-pog

assuming that was the intention of #37281 (comment)

cmsbuild · 2022-03-28T07:42:06Z

New categories assigned: tracking-pog

@mmusich,@mtosi,@vmariani you have been requested to review this Pull request/Issue and eventually sign? Thanks

mmusich · 2022-03-28T07:42:12Z

+1

@cms-sw/reconstruction-l2 what is being waited upon here?

jpata · 2022-03-28T11:26:00Z

Thanks Marco, I wanted to ask tracking to take a look (and make sure it was discussed in the POG) as I didn't really have any comments. I will sign now.

jpata · 2022-03-28T11:26:51Z

+reconstruction

fwyzard · 2022-03-28T11:52:20Z

RecoPixelVertexing/PixelTriplets/plugins/CAHitNtupletGeneratorKernelsImpl.h

+  auto ntracks = apc->get().m;
+  if (0 == first)
+    tracks.setNTracks(ntracks);
+  for (int idx = first, nt = ntracks; idx < nt; idx += gridDim.x * blockDim.x) {


just checking -- why is nt needed ? wouldn't idx < ntracks equivalent ?

fwyzard · 2022-03-28T11:53:36Z

+heterogeneous

cmsbuild · 2022-03-28T11:53:55Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

missirol · 2022-03-28T12:05:29Z

Default has changed, so expect small regressions.

@VinInn @fwyzard @silviodonato , should this PR be backported to 12_3_X? (since HLT will use that release at the start of Run 3)

silviodonato · 2022-03-28T13:17:33Z

I definitely support the request of a backport, if it is possible. It would be very good if we can start the data taking including this PR.

perrotta · 2022-03-28T13:27:55Z

+1

missirol · 2022-03-28T17:11:59Z

I definitely support the request of a backport, if it is possible. It would be very good if we can start the data taking including this PR.

Okay, @silviodonato.

Unless anyone objects to this (or goes ahead with the backport), I will open the backport PR to 12_3_X in the next days as per TSG's request. (cc: @Martin-Grunewald)

VinInn added 8 commits March 11, 2022 15:43

add layer and filter qual

57c2e1a

on cpu as well

e082b5d

use z as hit id

907510d

Merged DumpTK from repository VinInn with cms-merge-topic

6895f82

clampPT

f3339ee

introduce ntracks in soa

9a8ff0b

code format

2c19fa9

use ntracks

a375a54

cmsbuild added this to the CMSSW_12_4_X milestone Mar 20, 2022

cmsbuild added code-checks-pending heterogeneous-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Mar 20, 2022

cmsbuild added tests-started and removed tests-pending labels Mar 20, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 20, 2022

cmsbuild added tests-approved and removed tests-started labels Mar 21, 2022

VinInn mentioned this pull request Mar 23, 2022

bug fix in vertex selection of L2TauTagNNProducer #37291

Merged

cmsbuild added the tracking-pog-pending label Mar 28, 2022

cmsbuild added tracking-pog-approved and removed tracking-pog-pending labels Mar 28, 2022

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Mar 28, 2022

fwyzard reviewed Mar 28, 2022

View reviewed changes

cmsbuild added fully-signed heterogeneous-approved and removed pending-signatures heterogeneous-pending labels Mar 28, 2022

cmsbuild added orp-approved and removed orp-pending labels Mar 28, 2022

cmsbuild merged commit 2620d3b into cms-sw:master Mar 28, 2022

missirol mentioned this pull request Apr 2, 2022

Backport to 12_3_X of Patatrack pixel-reco updates (pT-clamping in vertex sorting + stabilized Fishbone) #37444

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Patatrack: introduce pt-clamping in vertex sorting, add #tracks in track SOA #37281

Patatrack: introduce pt-clamping in vertex sorting, add #tracks in track SOA #37281

VinInn commented Mar 20, 2022 •

edited

VinInn commented Mar 20, 2022

VinInn commented Mar 20, 2022

cmsbuild commented Mar 20, 2022

cmsbuild commented Mar 20, 2022

VinInn commented Mar 21, 2022

mmusich commented Mar 21, 2022

smuzaffar commented Mar 21, 2022

cmsbuild commented Mar 21, 2022

jpata commented Mar 22, 2022 •

edited

mmusich commented Mar 28, 2022

cmsbuild commented Mar 28, 2022

mmusich commented Mar 28, 2022

jpata commented Mar 28, 2022

jpata commented Mar 28, 2022

fwyzard Mar 28, 2022

fwyzard commented Mar 28, 2022

cmsbuild commented Mar 28, 2022

missirol commented Mar 28, 2022

silviodonato commented Mar 28, 2022

perrotta commented Mar 28, 2022

missirol commented Mar 28, 2022

Patatrack: introduce pt-clamping in vertex sorting, add #tracks in track SOA #37281

Patatrack: introduce pt-clamping in vertex sorting, add #tracks in track SOA #37281

Conversation

VinInn commented Mar 20, 2022 • edited

VinInn commented Mar 20, 2022

VinInn commented Mar 20, 2022

cmsbuild commented Mar 20, 2022

cmsbuild commented Mar 20, 2022

VinInn commented Mar 21, 2022

mmusich commented Mar 21, 2022

smuzaffar commented Mar 21, 2022

cmsbuild commented Mar 21, 2022

GPU Comparison Summary

Comparison Summary

jpata commented Mar 22, 2022 • edited

mmusich commented Mar 28, 2022

cmsbuild commented Mar 28, 2022

mmusich commented Mar 28, 2022

jpata commented Mar 28, 2022

jpata commented Mar 28, 2022

fwyzard Mar 28, 2022

Choose a reason for hiding this comment

fwyzard commented Mar 28, 2022

cmsbuild commented Mar 28, 2022

missirol commented Mar 28, 2022

silviodonato commented Mar 28, 2022

perrotta commented Mar 28, 2022

missirol commented Mar 28, 2022

VinInn commented Mar 20, 2022 •

edited

jpata commented Mar 22, 2022 •

edited