Follow up to 37617 #37798

AdrianoDee · 2022-05-04T10:55:44Z

PR description:

Follow up to #37617 :

fix issue highlighted in GPUvsCPU: updates to modules and workflows for pixel reconstruction #37617 (comment);
fix misalignment for .501 and .502 running on CPU (see GPUvsCPU: updates to modules and workflows for pixel reconstruction #37617 (comment)). The wfs are now identical (same naming, same results, see 501 and 502 on CPU). This is accomplished (when pixelNtupletFit modifier is on) by:
- when only pixelNtupletFit modifier is on:
  - adding siPixelRecHitsPreSplittingCPU, which is the actual producer on CPU, to the task;
  - replacing the siPixelRecHitsPreSplitting@cpu with and EDAlias for legacy formats;
  - adding siPixelRecHitsPreSplittingSoA as a SwitchProducerCUDA with only @cpu branch as an EDAlias for SoA formats;
- when also the gpu modifier is on:
  - adding siPixelRecHitsPreSplittingCUDA, which is the actual producer on GPU, to the task;
  - adding to the siPixelRecHitsPreSplitting@cuda branch the CUDA to legacy converter;
  - adding to the siPixelRecHitsPreSplitting@cuda branch the CUDA to SoA converter;
add GPU and CPU local reco validation in monitorpixelSoACompareSource (the CPUvsGPU will come in a later PR).

PR validation:

Running 39434.501 and 11634.x , with x=501, 502 and 503.

cmsbuild · 2022-05-04T11:03:57Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37798/29709

This PR adds an extra 16KB to repository

cmsbuild · 2022-05-04T11:04:16Z

A new Pull Request was created by @AdrianoDee for master.

It involves the following packages:

DQM/SiPixelPhase1Heterogeneous (dqm)
RecoLocalTracker/SiPixelRecHits (reconstruction)
RecoPixelVertexing/PixelTrackFitting (reconstruction)

@emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @clacaputo, @slava77, @jpata, @pmandrik, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@ferencek, @hdelanno, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @gpetruc, @OzAmram, @fioriNTU, @jandrea, @mtosi, @idebruyn, @mmusich, @dkotlins, @threus, @dgulhan, @tvami this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2022-05-04T11:12:47Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37798/29711

This PR adds an extra 16KB to repository

cmsbuild · 2022-05-04T11:13:11Z

Pull request #37798 was updated. @emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @clacaputo, @slava77, @jpata, @pmandrik, @micsucmed, @rvenditti can you please check and sign again.

AdrianoDee · 2022-05-04T11:13:26Z

test parameters:

enable_tests = gpu
workflows_gpu = 11634.503, 11634.502
workflows = 11634.501, 39434.501
relvals_opt= -w standard,highstats,pileup,generator,extendedgen,production,upgrade,cleanedupgrade,ged

AdrianoDee · 2022-05-04T11:13:38Z

please test

fwyzard · 2022-05-04T11:27:49Z

assign heterogeneous

cmsbuild · 2022-05-04T11:28:11Z

New categories assigned: heterogeneous

@fwyzard,@makortel you have been requested to review this Pull request/Issue and eventually sign? Thanks

cmsbuild · 2022-05-04T15:40:56Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6adb12/24443/summary.html
COMMIT: 0d65e7d
CMSSW: CMSSW_12_4_X_2022-05-04-1100/slc7_amd64_gcc10
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37798/24443/install.sh to create a dev area with all the needed externals and cmssw changes.

GPU Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6adb12/11634.503_TTbar_14TeV+2021_Patatrack_PixelOnlyGPU_Validation+TTbar_14TeV_TuneCP5_GenSim+Digi+RecoNano+HARVESTNano

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 6
DQMHistoTests: Total histograms compared: 51890
DQMHistoTests: Total failures: 47
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 51843
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 16063.242 KiB( 5 files compared)
DQMHistoSizes: changed ( 11634.503 ): 8031.621 KiB SiPixelHeterogeneous/PixelRecHitsSoACPU
DQMHistoSizes: changed ( 11634.503 ): 8031.621 KiB SiPixelHeterogeneous/PixelRecHitsSoAGPU
Checked 20 log files, 15 edm output root files, 6 DQM output files
TriggerResults: no differences found

Comparison Summary

There are some workflows for which there are errors in the baseline:
39434.501 step 3
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6adb12/39434.501_TTbar_14TeV+2026D88_Patatrack_PixelOnlyCPU+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3716560
DQMHistoTests: Total failures: 13
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3716524
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 49 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 212 log files, 51 edm output root files, 50 DQM output files
TriggerResults: no differences found

cmsbuild · 2022-05-11T12:09:26Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37798/29929

This PR adds an extra 20KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DQM/SiPixelPhase1Heterogeneous/python/SiPixelPhase1HeterogenousDQM_FirstStep_cff.py modified in PR(s): Pixel Heterogeneous DQM: add module rechit @cpu / @gpu comparisons #37860
- File RecoLocalTracker/SiPixelRecHits/python/SiPixelRecHits_cfi.py modified in PR(s): New Heterogeneous Memory Pool #37887

cmsbuild · 2022-05-11T12:09:48Z

Pull request #37798 was updated. @emanueleusai, @makortel, @fwyzard, @ahmad3213, @jfernan2, @clacaputo, @slava77, @jpata, @pmandrik, @micsucmed, @rvenditti can you please check and sign again.

fwyzard · 2022-05-11T12:12:47Z

+heterogeneous

cmsbuild · 2022-05-11T17:25:31Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6adb12/24620/summary.html
COMMIT: 67ee0e0
CMSSW: CMSSW_12_4_X_2022-05-10-2300/slc7_amd64_gcc10
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37798/24620/install.sh to create a dev area with all the needed externals and cmssw changes.

GPU Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6adb12/11634.503_TTbar_14TeV+2021_Patatrack_PixelOnlyGPU_Validation+TTbar_14TeV_TuneCP5_GenSim+Digi+RecoNano+HARVESTNano

Summary:

No significant changes to the logs found
Reco comparison results: 24 differences found in the comparisons
DQMHistoTests: Total files compared: 6
DQMHistoTests: Total histograms compared: 51890
DQMHistoTests: Total failures: 972
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 50918
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 16063.242 KiB( 5 files compared)
DQMHistoSizes: changed ( 11634.503 ): 8031.621 KiB SiPixelHeterogeneous/PixelRecHitsSoACPU
DQMHistoSizes: changed ( 11634.503 ): 8031.621 KiB SiPixelHeterogeneous/PixelRecHitsSoAGPU
Checked 20 log files, 15 edm output root files, 6 DQM output files
TriggerResults: no differences found

Comparison Summary

There are some workflows for which there are errors in the baseline:
39434.501 step 3
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6adb12/39434.501_TTbar_14TeV+2026D88_Patatrack_PixelOnlyCPU+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3728657
DQMHistoTests: Total failures: 2
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3728633
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
Checked 217 log files, 54 edm output root files, 51 DQM output files
TriggerResults: no differences found

qliphy · 2022-05-12T01:01:12Z

urgent
to fix IB issue and be included in 12_4_0_pre4

jpata · 2022-05-12T06:53:36Z

+reconstruction

since Follow up to 37617 #37798 (comment), removed the gpu modifier from pixels (fab1125)

perrotta · 2022-05-12T07:34:40Z

@emanueleusai could you please check this PR, and either confirm your previous signature or comment otherwise?

emanueleusai · 2022-05-12T16:02:56Z

+1

cmsbuild · 2022-05-12T16:03:18Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-05-12T16:15:12Z

+1

Follow up to 37617

d153bd3

cmsbuild added this to the CMSSW_12_4_X milestone May 4, 2022

cmsbuild added code-checks-pending dqm-pending orp-pending pending-signatures reconstruction-pending tests-pending labels May 4, 2022

AdrianoDee mentioned this pull request May 4, 2022

GPUvsCPU: updates to modules and workflows for pixel reconstruction #37617

Merged

cmsbuild added code-checks-approved and removed code-checks-pending labels May 4, 2022

Using both modifiers for GPU

0d65e7d

cmsbuild added code-checks-pending and removed code-checks-approved labels May 4, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels May 4, 2022

cmsbuild added tests-started and removed tests-pending labels May 4, 2022

cmsbuild added the heterogeneous-pending label May 4, 2022

cmsbuild added tests-approved and removed tests-started labels May 4, 2022

cmsbuild added code-checks-pending tests-started labels May 11, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels May 11, 2022

cmsbuild added heterogeneous-approved and removed heterogeneous-pending labels May 11, 2022

cmsbuild added tests-approved and removed tests-started labels May 11, 2022

cmsbuild added the urgent label May 12, 2022

cmsbuild added reconstruction-approved and removed reconstruction-pending labels May 12, 2022

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels May 12, 2022

cmsbuild added orp-approved and removed orp-pending labels May 12, 2022

cmsbuild merged commit 0f7bef7 into cms-sw:master May 12, 2022

AdrianoDee mentioned this pull request May 12, 2022

[12_3_X] Backport of 37617 + 37798 #37933

Merged

AdrianoDee deleted the gpu_dqm_pixels_124_Fix branch May 12, 2022 18:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Follow up to 37617 #37798

Follow up to 37617 #37798

AdrianoDee commented May 4, 2022 •

edited

cmsbuild commented May 4, 2022

cmsbuild commented May 4, 2022

cmsbuild commented May 4, 2022

cmsbuild commented May 4, 2022

AdrianoDee commented May 4, 2022

AdrianoDee commented May 4, 2022

fwyzard commented May 4, 2022

cmsbuild commented May 4, 2022

cmsbuild commented May 4, 2022

cmsbuild commented May 11, 2022

cmsbuild commented May 11, 2022

fwyzard commented May 11, 2022

cmsbuild commented May 11, 2022

qliphy commented May 12, 2022

jpata commented May 12, 2022

perrotta commented May 12, 2022

emanueleusai commented May 12, 2022

cmsbuild commented May 12, 2022

perrotta commented May 12, 2022

Follow up to 37617 #37798

Follow up to 37617 #37798

Conversation

AdrianoDee commented May 4, 2022 • edited

PR description:

PR validation:

cmsbuild commented May 4, 2022

cmsbuild commented May 4, 2022

cmsbuild commented May 4, 2022

cmsbuild commented May 4, 2022

AdrianoDee commented May 4, 2022

AdrianoDee commented May 4, 2022

fwyzard commented May 4, 2022

cmsbuild commented May 4, 2022

cmsbuild commented May 4, 2022

GPU Comparison Summary

Comparison Summary

cmsbuild commented May 11, 2022

cmsbuild commented May 11, 2022

fwyzard commented May 11, 2022

cmsbuild commented May 11, 2022

GPU Comparison Summary

Comparison Summary

qliphy commented May 12, 2022

jpata commented May 12, 2022

perrotta commented May 12, 2022

emanueleusai commented May 12, 2022

cmsbuild commented May 12, 2022

perrotta commented May 12, 2022

AdrianoDee commented May 4, 2022 •

edited