Lin log chi2 cut for patatrack #34419

VinInn · 2021-07-09T08:13:44Z

Modify the parameterization of the chi2cut to a simple linlog.
Chi2 cut "optimized" for PU ~50 using current realistic conditions for pixel detector.

Took the opportunity the address the issues raised in #34400

VinInn · 2021-07-09T08:13:56Z

@czangela

cmsbuild · 2021-07-09T08:21:26Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34419/23831

This PR adds an extra 40KB to repository

cmsbuild · 2021-07-09T08:21:47Z

A new Pull Request was created by @VinInn (Vincenzo Innocente) for master.

It involves the following packages:

CUDADataFormats/TrackingRecHit (heterogeneous, reconstruction)
RecoLocalTracker/SiPixelRecHits (reconstruction)
RecoPixelVertexing/PixelTriplets (reconstruction)

@perrotta, @makortel, @slava77, @cmsbuild, @fwyzard, @jpata can you please review it and eventually sign? Thanks.
@mtosi, @makortel, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @OzAmram, @ferencek, @dkotlins, @gpetruc, @mmusich, @threus, @dgulhan, @tvami this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy, @perrotta you are the release manager for this.

cms-bot commands are listed here

VinInn · 2021-07-09T08:38:39Z

@cmsbuild , please test

VinInn · 2021-07-09T08:38:49Z

enable gpu

cmsbuild · 2021-07-09T12:02:07Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a918ad/16640/summary.html
COMMIT: eb13d4b
CMSSW: CMSSW_12_0_X_2021-07-08-2300/slc7_amd64_gcc900
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34419/16640/install.sh to create a dev area with all the needed externals and cmssw changes.

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 12 differences found in the comparisons
DQMHistoTests: Total files compared: 4
DQMHistoTests: Total histograms compared: 19735
DQMHistoTests: Total failures: 3097
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 16638
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
Checked 12 log files, 9 edm output root files, 4 DQM output files
TriggerResults: no differences found

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 5130 differences found in the comparisons
DQMHistoTests: Total files compared: 38
DQMHistoTests: Total histograms compared: 2786260
DQMHistoTests: Total failures: 12
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2786225
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 37 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 160 log files, 37 edm output root files, 38 DQM output files
TriggerResults: no differences found

slava77 · 2021-07-09T12:35:04Z

@cmsbuild please test

to get cleaner CPU setup comparisons

VinInn · 2021-07-09T14:18:30Z

tested speed on V100
Running 4 times over 5000 events with 1 jobs, each with 8 threads, 8 streams and 1 GPUs
triplets GPU only
CMSSW_12_0_0_pre2: 1167.9 ± 2.9 ev/s
CMSSW_12_0_0_pre3: 1184.8 ± 6.0 ev/s
CMSSW_12_0_X_2021-06-30-2300: 1187.4 ± 10.3 ev/s
CMSSW_12_0_X_2021-07-08-2300: 1185.2 ± 16.0 ev/s
this PR: 1195.8 ± 8.3 ev/s

cmsbuild · 2021-07-09T15:58:57Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a918ad/16655/summary.html
COMMIT: eb13d4b
CMSSW: CMSSW_12_0_X_2021-07-09-0800/slc7_amd64_gcc900
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34419/16655/install.sh to create a dev area with all the needed externals and cmssw changes.

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 12 differences found in the comparisons
DQMHistoTests: Total files compared: 4
DQMHistoTests: Total histograms compared: 19735
DQMHistoTests: Total failures: 3113
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 16622
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
Checked 12 log files, 9 edm output root files, 4 DQM output files
TriggerResults: no differences found

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 38
DQMHistoTests: Total histograms compared: 2786260
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2786231
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 37 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 160 log files, 37 edm output root files, 38 DQM output files
TriggerResults: no differences found

VinInn · 2021-07-10T13:07:01Z

@cmsbuild please test

cmsbuild · 2021-07-10T13:12:56Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34419/23861

This PR adds an extra 8KB to repository

cmsbuild · 2021-07-10T13:13:24Z

Pull request #34419 was updated. @perrotta, @makortel, @fwyzard, @jpata, @slava77 can you please check and sign again.

cmsbuild · 2021-07-10T16:09:03Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a918ad/16678/summary.html
COMMIT: dcdaca5
CMSSW: CMSSW_12_0_X_2021-07-10-1100/slc7_amd64_gcc900
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34419/16678/install.sh to create a dev area with all the needed externals and cmssw changes.

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
Reco comparison had 3 failed jobs
DQMHistoTests: Total files compared: 4
DQMHistoTests: Total histograms compared: 0
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 0
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
Checked 12 log files, 9 edm output root files, 4 DQM output files
TriggerResults: no differences found

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 38
DQMHistoTests: Total histograms compared: 0
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 0
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 37 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 160 log files, 37 edm output root files, 38 DQM output files
TriggerResults: no differences found

VinInn · 2021-07-11T10:50:51Z

Timing on T4 (empty machine, fresh reboot)

CMSSW_12_0_0_pre2: 713.8 ± 1.6 ev/s
CMSSW_12_0_0_pre3: 710.1 ± 2.7 ev/s
CMSSW_12_0_X_2021-07-06-2300: 749.5 ± 2.8 ev/s
CMSSW_12_0_0_pre4: 748.1 ± 2.0 ev/s
this PR: 748.5 ± 1.6 ev/s

slava77 · 2021-07-15T12:20:31Z

+reconstruction

for #34419 dcdaca5

code changes are in line with the PR description
- items from follow up to PR review of CPEFast to better reproduce Generic #34286 #34400 are mostly/partially addressed
jenkins tests pass and comparisons with the baseline show small differences in the pixel tracking GPU workflow

fwyzard · 2021-07-16T08:11:32Z

+heterogeneous

cmsbuild · 2021-07-16T08:11:55Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy, @perrotta (and backports should be raised in the release meeting by the corresponding L2)

fwyzard · 2021-07-16T08:12:06Z

out of curiosity: how does the performance of roughLog compare with CUDA native ::logf ?

fwyzard · 2021-07-16T08:16:18Z

there is also a fast approximate log2 PTX instruction, lg2.approx.ftz.f32, but I've never tried using inline PTX and I don't know if the CUDA ::log2f uses that or not.

qliphy · 2021-07-18T02:40:58Z

+1

qliphy · 2021-07-18T02:41:27Z

out of curiosity: how does the performance of roughLog compare with CUDA native ::logf ?

@VinInn Do you have numbers for the question?

VinInn added 4 commits July 9, 2021 08:50

introduce new chi2cut params

41b54b0

syntax fix

e3d50f8

add comments. clarify code

e98cef6

remove cout

eb13d4b

cmsbuild added this to the CMSSW_12_0_X milestone Jul 9, 2021

cmsbuild added code-checks-pending heterogeneous-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Jul 9, 2021

VinInn mentioned this pull request Jul 9, 2021

follow up to PR review of CPEFast to better reproduce Generic #34286 #34400

Open

7 tasks

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 9, 2021

cmsbuild added tests-started and removed tests-pending labels Jul 9, 2021

VinInn changed the title ~~Lin log chi2 cut~~ Lin log chi2 cut for patatrack Jul 9, 2021

cmsbuild added tests-approved and removed tests-started labels Jul 9, 2021

cmsbuild added tests-started and removed tests-approved labels Jul 9, 2021

cmsbuild added the tests-pending label Jul 10, 2021

cmsbuild added tests-started and removed tests-pending labels Jul 10, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 10, 2021

cmsbuild added tests-approved and removed tests-started labels Jul 10, 2021

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Jul 15, 2021

cmsbuild mentioned this pull request Jul 15, 2021

Pixel Local GPU reco crash on copy fix #34501

Merged

cmsbuild added fully-signed heterogeneous-approved and removed heterogeneous-pending pending-signatures labels Jul 16, 2021

cmsbuild added orp-approved and removed orp-pending labels Jul 18, 2021

cmsbuild merged commit b8709ea into cms-sw:master Jul 18, 2021

cmsbuild mentioned this pull request Jul 19, 2021

[BuildRules] cleanup PYTHON27; use python3 as default cms-sw/cmsdist#7146

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Lin log chi2 cut for patatrack #34419

Lin log chi2 cut for patatrack #34419

VinInn commented Jul 9, 2021 •

edited

VinInn commented Jul 9, 2021

cmsbuild commented Jul 9, 2021

cmsbuild commented Jul 9, 2021

VinInn commented Jul 9, 2021

VinInn commented Jul 9, 2021

cmsbuild commented Jul 9, 2021

slava77 commented Jul 9, 2021

VinInn commented Jul 9, 2021 •

edited

cmsbuild commented Jul 9, 2021

VinInn commented Jul 10, 2021

cmsbuild commented Jul 10, 2021

cmsbuild commented Jul 10, 2021

cmsbuild commented Jul 10, 2021

VinInn commented Jul 11, 2021 •

edited

slava77 commented Jul 15, 2021

fwyzard commented Jul 16, 2021

cmsbuild commented Jul 16, 2021

fwyzard commented Jul 16, 2021

fwyzard commented Jul 16, 2021

qliphy commented Jul 18, 2021

qliphy commented Jul 18, 2021

Lin log chi2 cut for patatrack #34419

Lin log chi2 cut for patatrack #34419

Conversation

VinInn commented Jul 9, 2021 • edited

VinInn commented Jul 9, 2021

cmsbuild commented Jul 9, 2021

cmsbuild commented Jul 9, 2021

VinInn commented Jul 9, 2021

VinInn commented Jul 9, 2021

cmsbuild commented Jul 9, 2021

GPU Comparison Summary

Comparison Summary

slava77 commented Jul 9, 2021

VinInn commented Jul 9, 2021 • edited

cmsbuild commented Jul 9, 2021

GPU Comparison Summary

Comparison Summary

VinInn commented Jul 10, 2021

cmsbuild commented Jul 10, 2021

cmsbuild commented Jul 10, 2021

cmsbuild commented Jul 10, 2021

GPU Comparison Summary

Comparison Summary

VinInn commented Jul 11, 2021 • edited

slava77 commented Jul 15, 2021

fwyzard commented Jul 16, 2021

cmsbuild commented Jul 16, 2021

fwyzard commented Jul 16, 2021

fwyzard commented Jul 16, 2021

qliphy commented Jul 18, 2021

qliphy commented Jul 18, 2021

VinInn commented Jul 9, 2021 •

edited

VinInn commented Jul 9, 2021 •

edited

VinInn commented Jul 11, 2021 •

edited