ECAL GPU unpacker: adapt the buffer size to the ECAL FEDs size [12.3.x] #38205

fwyzard · 2022-06-02T15:40:32Z

PR description:

Automatically determine the cpu and gpu buffers' size used for the unpacking from the total size of the non-empty ECAL FEDs.
Remove the maxFedSize parameter from the EcalRawToDigi modules in the HLT menus.

PR validation:

None.

if this PR is a backport please specify the original PR and why you need to backport that PR:

Backport of #38202 .

fwyzard · 2022-06-02T15:40:39Z

type bugfix

cmsbuild · 2022-06-02T15:40:54Z

A new Pull Request was created by @fwyzard (Andrea Bocci) for CMSSW_12_3_X.

It involves the following packages:

EventFilter/EcalRawToDigi (reconstruction)
HLTrigger/Configuration (hlt)

@Martin-Grunewald, @clacaputo, @cmsbuild, @missirol, @slava77, @jpata can you please review it and eventually sign? Thanks.
@rchatter, @argiro, @Martin-Grunewald, @missirol, @silviodonato, @thomreis, @simonepigazzini this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

Backported from ECAL GPU unpacker: adapt the buffer size to the ECAL FEDs size #38202

fwyzard · 2022-06-02T15:40:55Z

backport #38202

fwyzard · 2022-06-02T15:41:05Z

enable gpu

fwyzard · 2022-06-02T15:41:08Z

please test

cmsbuild · 2022-06-02T18:47:31Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-fc0203/25210/summary.html
COMMIT: 97e7f2b
CMSSW: CMSSW_12_3_X_2022-06-02-1100/slc7_amd64_gcc10
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38205/25210/install.sh to create a dev area with all the needed externals and cmssw changes.

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 24 differences found in the comparisons
DQMHistoTests: Total files compared: 4
DQMHistoTests: Total histograms compared: 19864
DQMHistoTests: Total failures: 997
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 18867
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
Checked 12 log files, 9 edm output root files, 4 DQM output files
TriggerResults: no differences found

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3627388
DQMHistoTests: Total failures: 2
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3627364
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 205 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

missirol · 2022-06-05T17:14:40Z

This branch has conflicts that must be resolved

@fwyzard , unfortunately this PR needs to be rebased (the conflict comes from #38124).

Automatically determine the cpu and gpu buffers' size used for the unpacking from the total size of the non-empty ECAL FEDs.

fwyzard · 2022-06-06T22:36:53Z

Rebased to fix the merge conflict.

fwyzard · 2022-06-06T22:36:56Z

enable gpu

cmsbuild · 2022-06-06T22:37:00Z

Pull request #38205 was updated. @Martin-Grunewald, @clacaputo, @cmsbuild, @missirol, @slava77, @jpata can you please check and sign again.

fwyzard · 2022-06-06T22:37:03Z

please test

cmsbuild · 2022-06-07T06:20:11Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-fc0203/25326/summary.html
COMMIT: 7ffb562
CMSSW: CMSSW_12_3_X_2022-06-06-2300/slc7_amd64_gcc10
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38205/25326/install.sh to create a dev area with all the needed externals and cmssw changes.

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 24 differences found in the comparisons
DQMHistoTests: Total files compared: 4
DQMHistoTests: Total histograms compared: 19864
DQMHistoTests: Total failures: 996
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 18868
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
Checked 12 log files, 9 edm output root files, 4 DQM output files
TriggerResults: no differences found

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3628978
DQMHistoTests: Total failures: 13
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3628942
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 48 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 205 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

missirol · 2022-06-07T06:27:35Z

HLTrigger/Configuration/python/customizeHLTforCMSSW.py

@@ -189,12 +189,23 @@ def customiseForOffline(process):
    return process


+# ECAL GPU unpacker: adapt the buffer size to the ECAL FEDs size (#38202)
+# remove the EcalRawToDigi.maxFedSize parameter from the menu
+def customizeHLTfor38202(process):


Should be 38205, but I don't object, it's clear enough.

missirol · 2022-06-07T06:27:51Z

+hlt

clacaputo · 2022-06-08T17:27:41Z

There are some reco infinitesimal differences in GPU comparison that weren't present in #38204 and #38202 .
More specifically, there is an additional entry in recoTracks_pixelTracks__RECO_obj, that doesn't change any other quantity downstream. You can also see the same effect in DQM https://tinyurl.com/2698w9o6
@fwyzard , do you happen to know if it's just a fluctuation in GPU workflows?

fwyzard · 2022-06-08T18:10:47Z

@fwyzard , do you happen to know if it's just a fluctuation in GPU workflows?

Mhm, given that this PR touches only the ECAL unpacker and nothing Pixel-related, I would assume so.

clacaputo · 2022-06-09T07:43:35Z

+reconstruction

this PR is a backport of ECAL GPU unpacker: adapt the buffer size to the ECAL FEDs size #38202 and its content is in line with the original pr
minor reco differences in gpu worflow, non related to the PR

cmsbuild · 2022-06-09T07:44:01Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_3_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_5_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2022-06-09T08:11:55Z

+1

cmsbuild added this to the CMSSW_12_3_X milestone Jun 2, 2022

cmsbuild added bug-fix hlt-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Jun 2, 2022

cmsbuild added the backport label Jun 2, 2022

cmsbuild added tests-started and removed tests-pending labels Jun 2, 2022

cmsbuild added tests-approved and removed tests-started labels Jun 2, 2022

fwyzard added 2 commits June 7, 2022 00:36

Adapt the buffer size to the ECAL FEDs size

7d865d1

Automatically determine the cpu and gpu buffers' size used for the unpacking from the total size of the non-empty ECAL FEDs.

Remove the EcalRawToDigi.maxFedSize parameter from the HLT menu

7ffb562

fwyzard force-pushed the automatic_ECAL_buffer_size_123x branch from 97e7f2b to 7ffb562 Compare June 6, 2022 22:36

cmsbuild added tests-pending and removed tests-approved labels Jun 6, 2022

cmsbuild added tests-started and removed tests-pending labels Jun 6, 2022

cmsbuild added tests-approved and removed tests-started labels Jun 7, 2022

missirol reviewed Jun 7, 2022

View reviewed changes

cmsbuild added hlt-approved backport-ok and removed hlt-pending backport labels Jun 7, 2022

cmsbuild added fully-signed reconstruction-approved and removed reconstruction-pending pending-signatures labels Jun 9, 2022

cmsbuild added orp-approved and removed orp-pending labels Jun 9, 2022

cmsbuild merged commit 2b12642 into cms-sw:CMSSW_12_3_X Jun 9, 2022

fwyzard deleted the automatic_ECAL_buffer_size_123x branch July 31, 2022 13:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ECAL GPU unpacker: adapt the buffer size to the ECAL FEDs size [12.3.x] #38205

ECAL GPU unpacker: adapt the buffer size to the ECAL FEDs size [12.3.x] #38205

fwyzard commented Jun 2, 2022

fwyzard commented Jun 2, 2022

cmsbuild commented Jun 2, 2022 •

edited

fwyzard commented Jun 2, 2022

fwyzard commented Jun 2, 2022

fwyzard commented Jun 2, 2022

cmsbuild commented Jun 2, 2022

missirol commented Jun 5, 2022

fwyzard commented Jun 6, 2022

fwyzard commented Jun 6, 2022

cmsbuild commented Jun 6, 2022

fwyzard commented Jun 6, 2022

cmsbuild commented Jun 7, 2022

missirol Jun 7, 2022

missirol commented Jun 7, 2022

clacaputo commented Jun 8, 2022

fwyzard commented Jun 8, 2022

clacaputo commented Jun 9, 2022

cmsbuild commented Jun 9, 2022

qliphy commented Jun 9, 2022

Navigation Menu

ECAL GPU unpacker: adapt the buffer size to the ECAL FEDs size [12.3.x] #38205

ECAL GPU unpacker: adapt the buffer size to the ECAL FEDs size [12.3.x] #38205

Conversation

fwyzard commented Jun 2, 2022

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

fwyzard commented Jun 2, 2022

cmsbuild commented Jun 2, 2022 • edited

fwyzard commented Jun 2, 2022

fwyzard commented Jun 2, 2022

fwyzard commented Jun 2, 2022

cmsbuild commented Jun 2, 2022

GPU Comparison Summary

Comparison Summary

missirol commented Jun 5, 2022

fwyzard commented Jun 6, 2022

fwyzard commented Jun 6, 2022

cmsbuild commented Jun 6, 2022

fwyzard commented Jun 6, 2022

cmsbuild commented Jun 7, 2022

GPU Comparison Summary

Comparison Summary

missirol Jun 7, 2022

Choose a reason for hiding this comment

missirol commented Jun 7, 2022

clacaputo commented Jun 8, 2022

fwyzard commented Jun 8, 2022

clacaputo commented Jun 9, 2022

cmsbuild commented Jun 9, 2022

qliphy commented Jun 9, 2022

cmsbuild commented Jun 2, 2022 •

edited