Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for the gpu modifier at HLT [12.0.x] #34979

Merged
merged 3 commits into from Sep 23, 2021

Conversation

fwyzard
Copy link
Contributor

@fwyzard fwyzard commented Aug 23, 2021

PR description:

When the gpu modifier is enabled, make the Pixel, ECAL and HCAL reconstruction running at HLT offloadable to a GPU.

PR validation:

Tested with

cmsDriver.py reHLT \
    --processName reHLT \
    -s HLT:@relval2021 \
    --conditions 120X_mcRun3_2021_realistic_v4 \
    --datatier GEN-SIM-DIGI-RAW \
    -n 10 \
    --eventcontent FEVTDEBUGHLT \
    --geometry DB:Extended \
    --era Run3 \
    --procModifiers gpu \
    --filein /store/relval/CMSSW_12_0_0_pre6/RelValTTbar_14TeV/GEN-SIM-DIGI-RAW/PU_120X_mcRun3_2021_realistic_v4_JIRA_129-v1/00000/79c06ed5-929b-4a57-a4f2-1ae90e6b38c5.root

With this change, the customizeHLTforPatatrack() customisation will be tested as part of the .502, .512 and .522 GPU workflows, and customizeHLTforPatatrackTriplets() with the .506 workflow.

if this PR is a backport please specify the original PR and why you need to backport that PR:

Backport to 12.0.0 of #34978.

@fwyzard fwyzard changed the base branch from master to CMSSW_12_0_X August 23, 2021 11:32
@cmsbuild cmsbuild added this to the CMSSW_12_0_X milestone Aug 23, 2021
@cmsbuild
Copy link
Contributor

A new Pull Request was created by @fwyzard (Andrea Bocci) for CMSSW_12_0_X.

It involves the following packages:

  • HLTrigger/Configuration (hlt)

@Martin-Grunewald, @cmsbuild can you please review it and eventually sign? Thanks.
@Martin-Grunewald this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

@fwyzard
Copy link
Contributor Author

fwyzard commented Aug 23, 2021

enable gpu

@fwyzard
Copy link
Contributor Author

fwyzard commented Aug 23, 2021

please test

@fwyzard fwyzard changed the title Hlt customisation for gpu 120x Add support for the gpu modifier at HLT (12.0.x) Aug 23, 2021
@fwyzard fwyzard changed the title Add support for the gpu modifier at HLT (12.0.x) Add support for the gpu modifier at HLT [12.0.x] Aug 23, 2021
@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3ce193/17965/summary.html
COMMIT: 66dce51
CMSSW: CMSSW_12_0_X_2021-08-23-1100/slc7_amd64_gcc900
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34979/17965/install.sh to create a dev area with all the needed externals and cmssw changes.

GPU Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 4
  • DQMHistoTests: Total histograms compared: 19735
  • DQMHistoTests: Total failures: 6
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 19729
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
  • Checked 12 log files, 9 edm output root files, 4 DQM output files
  • TriggerResults: no differences found

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 4 differences found in the comparisons
  • DQMHistoTests: Total files compared: 39
  • DQMHistoTests: Total histograms compared: 2998564
  • DQMHistoTests: Total failures: 5
  • DQMHistoTests: Total nulls: 1
  • DQMHistoTests: Total successes: 2998536
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -0.004 KiB( 38 files compared)
  • DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
  • Checked 165 log files, 37 edm output root files, 39 DQM output files
  • TriggerResults: no differences found

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 8, 2021

Pull request #34979 was updated. @cmsbuild, @jordan-martins, @chayanit, @bbilin, @wajidalikhan, @Martin-Grunewald, @AdrianoDee, @srimanob, @kskovpen can you please check and sign again.

@fwyzard
Copy link
Contributor Author

fwyzard commented Sep 8, 2021

Rebased on top of CMSSW_12_0_0, and added support for running the HLT step on GPU in runTheMatrix.

@fwyzard
Copy link
Contributor Author

fwyzard commented Sep 8, 2021

enable gpu

@fwyzard
Copy link
Contributor Author

fwyzard commented Sep 8, 2021

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 9, 2021

-1

Failed Tests: RelVals RelVals-GPU RelVals-INPUT AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3ce193/18421/summary.html
COMMIT: be76577
CMSSW: CMSSW_12_0_X_2021-09-08-1100/slc7_amd64_gcc900
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34979/18421/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

  • 4.534.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT/step2_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT.log
  • 136.731136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2/step2_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log
  • 136.793136.793_RunDoubleEG2017C+RunDoubleEG2017C+HLTDR2_2017+RECODR2_2017reHLT_skimDoubleEG_Prompt+HARVEST2017/step2_RunDoubleEG2017C+RunDoubleEG2017C+HLTDR2_2017+RECODR2_2017reHLT_skimDoubleEG_Prompt+HARVEST2017.log
Expand to see more relval errors ...

RelVals-GPU

  • 10824.52210824.522_TTbar_13+2018_Patatrack_HCALOnlyGPU+TTbar_13TeV_TuneCUETP8M1_GenSim+Digi+RecoFakeHLT+HARVESTFakeHLT/step2_TTbar_13+2018_Patatrack_HCALOnlyGPU+TTbar_13TeV_TuneCUETP8M1_GenSim+Digi+RecoFakeHLT+HARVESTFakeHLT.log
  • 10824.50210824.502_TTbar_13+2018_Patatrack_PixelOnlyGPU+TTbar_13TeV_TuneCUETP8M1_GenSim+Digi+RecoFakeHLT+HARVESTFakeHLT/step2_TTbar_13+2018_Patatrack_PixelOnlyGPU+TTbar_13TeV_TuneCUETP8M1_GenSim+Digi+RecoFakeHLT+HARVESTFakeHLT.log
  • 10824.51210824.512_TTbar_13+2018_Patatrack_ECALOnlyGPU+TTbar_13TeV_TuneCUETP8M1_GenSim+Digi+RecoFakeHLT+HARVESTFakeHLT/step2_TTbar_13+2018_Patatrack_ECALOnlyGPU+TTbar_13TeV_TuneCUETP8M1_GenSim+Digi+RecoFakeHLT+HARVESTFakeHLT.log

RelVals-INPUT

  • 4.174.17_RunMinBias2011A+RunMinBias2011A+HLTD+RECODR1reHLT+HARVESTDR1reHLT+SKIMDreHLT/step2_RunMinBias2011A+RunMinBias2011A+HLTD+RECODR1reHLT+HARVESTDR1reHLT+SKIMDreHLT.log
  • 4.264.26_ZMuSkim2011A+ZMuSkim2011A+HLTDSKIM+RECODR1reHLT+HARVESTDR1reHLT/step2_ZMuSkim2011A+ZMuSkim2011A+HLTDSKIM+RECODR1reHLT+HARVESTDR1reHLT.log
  • 4.274.27_ZElSkim2011A+ZElSkim2011A+HLTDSKIM+RECODR1reHLT+HARVESTDR1reHLT/step2_ZElSkim2011A+ZElSkim2011A+HLTDSKIM+RECODR1reHLT+HARVESTDR1reHLT.log
Expand to see more relval errors ...

AddOn Tests

----- Begin Fatal Exception 09-Sep-2021 02:13:13 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named /build/cmsbld/jenkins/workspace/ib-run-pr-addon/CMSSW_12_0_X_2021-09-08-1100/src/HLTrigger/Configuration/test/OnLine_HLT_Fake.py
Exception Message:
 unknown python problem occurred.
ImportError: cannot import name 'customizeHLTforPatatrackQuadruplets' from 'HLTrigger.Configuration.customizeHLTforPatatrack' (/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34979/18421/CMSSW_12_0_X_2021-09-08-1100/python/HLTrigger/Configuration/customizeHLTforPatatrack.py)

At:
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34979/18421/CMSSW_12_0_X_2021-09-08-1100/python/HLTrigger/Configuration/customizeHLTforCMSSW.py(140): customizeHLTforCMSSW
  /build/cmsbld/jenkins/workspace/ib-run-pr-addon/CMSSW_12_0_X_2021-09-08-1100/src/HLTrigger/Configuration/test/OnLine_HLT_Fake.py(380): <module>

----- End Fatal Exception -------------------------------------------------
  • hlt_mc_FakecmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run1_mc_Fake --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein file:RelVal_Raw_Fake_MC.root --fileout file:RelVal_Raw_Fake_MC_HLT_RECO.root : FAILED - time: date Thu Sep 9 02:13:16 2021-date Thu Sep 9 02:09:34 2021 s - exit: 256
  • hlt_mc_Fake1
----- Begin Fatal Exception 09-Sep-2021 02:13:31 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named /build/cmsbld/jenkins/workspace/ib-run-pr-addon/CMSSW_12_0_X_2021-09-08-1100/src/HLTrigger/Configuration/test/OnLine_HLT_Fake1.py
Exception Message:
 unknown python problem occurred.
ImportError: cannot import name 'customizeHLTforPatatrackQuadruplets' from 'HLTrigger.Configuration.customizeHLTforPatatrack' (/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34979/18421/CMSSW_12_0_X_2021-09-08-1100/python/HLTrigger/Configuration/customizeHLTforPatatrack.py)

At:
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34979/18421/CMSSW_12_0_X_2021-09-08-1100/python/HLTrigger/Configuration/customizeHLTforCMSSW.py(140): customizeHLTforCMSSW
  /build/cmsbld/jenkins/workspace/ib-run-pr-addon/CMSSW_12_0_X_2021-09-08-1100/src/HLTrigger/Configuration/test/OnLine_HLT_Fake1.py(397): <module>

----- End Fatal Exception -------------------------------------------------
Expand to see more addon errors ...

Simplify the Pixel local reconstruction running on GPU, taking advatage of cms-sw#34259.
@fwyzard fwyzard force-pushed the HLT_customisation_for_GPU_120x branch from 0faf487 to d98f14e Compare September 15, 2021 16:26
@cmsbuild
Copy link
Contributor

Pull request #34979 was updated. @cmsbuild, @jordan-martins, @chayanit, @bbilin, @wajidalikhan, @Martin-Grunewald, @AdrianoDee, @srimanob, @kskovpen can you please check and sign again.

@fwyzard
Copy link
Contributor Author

fwyzard commented Sep 16, 2021

please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3ce193/18651/summary.html
COMMIT: d98f14e
CMSSW: CMSSW_12_0_X_2021-09-15-2300/slc7_amd64_gcc900
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34979/18651/install.sh to create a dev area with all the needed externals and cmssw changes.

GPU Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 4
  • DQMHistoTests: Total histograms compared: 19735
  • DQMHistoTests: Total failures: 6
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 19729
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
  • Checked 12 log files, 9 edm output root files, 4 DQM output files
  • TriggerResults: no differences found

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 8 differences found in the comparisons
  • DQMHistoTests: Total files compared: 39
  • DQMHistoTests: Total histograms compared: 2998564
  • DQMHistoTests: Total failures: 12
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2998530
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
  • Checked 165 log files, 37 edm output root files, 39 DQM output files
  • TriggerResults: no differences found

@Martin-Grunewald
Copy link
Contributor

+1

@srimanob
Copy link
Contributor

+Upgrade

@qliphy
Copy link
Contributor

qliphy commented Sep 23, 2021

@cms-sw/pdmv-l2 Any comment?

@bbilin
Copy link
Contributor

bbilin commented Sep 23, 2021

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

@qliphy
Copy link
Contributor

qliphy commented Sep 23, 2021

+1

@cmsbuild cmsbuild merged commit f69f62a into cms-sw:CMSSW_12_0_X Sep 23, 2021
@fwyzard fwyzard deleted the HLT_customisation_for_GPU_120x branch January 30, 2024 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants