Add helper functions for alpaka kernels [12.5.x] #39547

fwyzard · 2022-09-30T04:36:05Z

PR description:

Add helper functions for:

performing an integer division with rounding up, or rounding a value to the next multiple of an integer;
creating an efficient work division, assigning the number of work items either to the number of GPU threads or CPU elements;
perform a scalar or N-dimensional strided loop within a kernel, based on the grid size and number of elements.

Update the alpaka tests to use the new helper functions.

PR validation:

Unit tests pass.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Backport of #39546 to CMSSW_12_5_X for the Alpaka migration.

cmsbuild · 2022-09-30T04:36:26Z

A new Pull Request was created by @fwyzard (Andrea Bocci) for CMSSW_12_5_X.

It involves the following packages:

CUDADataFormats/PortableTestObjects (heterogeneous)
DataFormats/PortableTestObjects (heterogeneous)
DataFormats/SoATemplate (heterogeneous)
HeterogeneousCore/AlpakaCore (heterogeneous)
HeterogeneousCore/AlpakaInterface (heterogeneous)
HeterogeneousCore/AlpakaTest (heterogeneous)
HeterogeneousCore/CUDATest (heterogeneous)

@cmsbuild, @makortel, @fwyzard can you please review it and eventually sign? Thanks.
@makortel, @missirol, @rovere this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

Backported from Add helper functions for alpaka kernels #39546

fwyzard · 2022-09-30T04:36:30Z

backport #39546

fwyzard · 2022-09-30T04:36:35Z

enable gpu

fwyzard · 2022-09-30T04:36:56Z

please test with cms-sw/cmsdist#8107

cmsbuild · 2022-09-30T05:07:29Z

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a77b26/27869/summary.html
COMMIT: 58de740
CMSSW: CMSSW_12_5_X_2022-09-29-2300/el8_amd64_gcc10
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39547/27869/install.sh to create a dev area with all the needed externals and cmssw changes.

fwyzard · 2022-09-30T05:19:32Z

please test with cms-sw/cmsdist#8108

cmsbuild · 2022-09-30T10:42:26Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a77b26/27871/summary.html
COMMIT: 58de740
CMSSW: CMSSW_12_5_X_2022-09-29-2300/el8_amd64_gcc10
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39547/27871/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3699454
DQMHistoTests: Total failures: 13
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3699418
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 50 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 212 log files, 49 edm output root files, 51 DQM output files
TriggerResults: no differences found

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
Reco comparison had 3 failed jobs
DQMHistoTests: Total files compared: 4
DQMHistoTests: Total histograms compared: 19876
DQMHistoTests: Total failures: 8
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 19868
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
Checked 12 log files, 9 edm output root files, 4 DQM output files
TriggerResults: no differences found

cmsbuild · 2022-09-30T20:57:37Z

Pull request #39547 was updated. @cmsbuild, @makortel, @fwyzard can you please check and sign again.

fwyzard · 2022-09-30T20:57:48Z

please test with cms-sw/cmsdist#8108

cmsbuild · 2022-10-01T08:32:52Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a77b26/27896/summary.html
COMMIT: de68e6d
CMSSW: CMSSW_12_5_X_2022-09-30-2300/el8_amd64_gcc10
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39547/27896/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test testHeterogeneousCoreAlpakaTestWriteRead had ERRORS

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3699454
DQMHistoTests: Total failures: 8
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3699424
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
Checked 212 log files, 49 edm output root files, 51 DQM output files
TriggerResults: no differences found

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
Reco comparison had 3 failed jobs
DQMHistoTests: Total files compared: 4
DQMHistoTests: Total histograms compared: 19876
DQMHistoTests: Total failures: 10
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 19866
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
Checked 12 log files, 9 edm output root files, 4 DQM output files
TriggerResults: no differences found

Add helper functions for: - performing an integer division with rounding up, or rounding a value to the next multiple of an integer; - creating an efficient work division, assigning the number of work items either to the number of GPU threads or CPU elements; - perform a scalar or N-dimensional strided loop within a kernel, based on the grid size and number of elements.

fwyzard · 2022-10-03T16:18:20Z

enable gpu

fwyzard · 2022-10-03T16:18:24Z

please test

cmsbuild · 2022-10-03T16:18:35Z

Pull request #39547 was updated. @makortel, @fwyzard can you please check and sign again.

fwyzard · 2022-10-03T16:30:54Z

+heterogeneous

cmsbuild · 2022-10-03T16:31:15Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_5_X IBs after it passes the integration tests and once validation in the development release cycle CMSSW_12_6_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

fwyzard · 2022-10-03T22:04:58Z

I assume the relvals/input failures are unrelated to these changes ?

fwyzard · 2022-10-03T22:05:02Z

please test

cmsbuild · 2022-10-04T03:00:51Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a77b26/27956/summary.html
COMMIT: 355d9d3
CMSSW: CMSSW_12_5_X_2022-10-03-1100/el8_amd64_gcc10
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39547/27956/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3699454
DQMHistoTests: Total failures: 2
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3699430
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
Checked 212 log files, 49 edm output root files, 51 DQM output files
TriggerResults: no differences found

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
Reco comparison had 3 failed jobs
DQMHistoTests: Total files compared: 4
DQMHistoTests: Total histograms compared: 19876
DQMHistoTests: Total failures: 119
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 19757
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
Checked 12 log files, 9 edm output root files, 4 DQM output files
TriggerResults: no differences found

perrotta · 2022-10-04T18:59:26Z

+1

cmsbuild added this to the CMSSW_12_5_X milestone Sep 30, 2022

cmsbuild added heterogeneous-pending orp-pending pending-signatures tests-pending labels Sep 30, 2022

cmsbuild added the backport label Sep 30, 2022

cmsbuild added requires-external tests-started and removed tests-pending labels Sep 30, 2022

cmsbuild added tests-rejected and removed tests-started labels Sep 30, 2022

cmsbuild added tests-started and removed tests-rejected labels Sep 30, 2022

cmsbuild added tests-approved and removed tests-started labels Sep 30, 2022

fwyzard force-pushed the alpaka_framework_part2 branch from 58de740 to 05286d1 Compare September 30, 2022 20:57

cmsbuild added tests-pending and removed tests-approved requires-external labels Sep 30, 2022

cmsbuild added requires-external and removed tests-pending labels Sep 30, 2022

cmsbuild added the tests-started label Oct 1, 2022

cmsbuild added tests-rejected and removed tests-started labels Oct 1, 2022

fwyzard added 2 commits October 3, 2022 18:16

Update the alpaka tests to use the new helper functions

355d9d3

fwyzard force-pushed the alpaka_framework_part2 branch from de68e6d to 355d9d3 Compare October 3, 2022 16:18

cmsbuild added tests-started and removed tests-rejected labels Oct 3, 2022

cmsbuild added fully-signed heterogeneous-approved and removed pending-signatures heterogeneous-pending labels Oct 3, 2022

cmsbuild added tests-approved backport-ok and removed tests-started backport labels Oct 4, 2022

cmsbuild added orp-approved and removed orp-pending labels Oct 4, 2022

cmsbuild merged commit 980f9a3 into cms-sw:CMSSW_12_5_X Oct 4, 2022

fwyzard deleted the alpaka_framework_part2 branch October 11, 2022 14:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add helper functions for alpaka kernels [12.5.x] #39547

Add helper functions for alpaka kernels [12.5.x] #39547

fwyzard commented Sep 30, 2022

cmsbuild commented Sep 30, 2022 •

edited

Loading

fwyzard commented Sep 30, 2022

fwyzard commented Sep 30, 2022

fwyzard commented Sep 30, 2022

cmsbuild commented Sep 30, 2022

fwyzard commented Sep 30, 2022

cmsbuild commented Sep 30, 2022

cmsbuild commented Sep 30, 2022

fwyzard commented Sep 30, 2022

cmsbuild commented Oct 1, 2022

fwyzard commented Oct 3, 2022

fwyzard commented Oct 3, 2022

cmsbuild commented Oct 3, 2022

fwyzard commented Oct 3, 2022

cmsbuild commented Oct 3, 2022

fwyzard commented Oct 3, 2022

fwyzard commented Oct 3, 2022

cmsbuild commented Oct 4, 2022

perrotta commented Oct 4, 2022

Add helper functions for alpaka kernels [12.5.x] #39547

Add helper functions for alpaka kernels [12.5.x] #39547

Conversation

fwyzard commented Sep 30, 2022

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Sep 30, 2022 • edited Loading

fwyzard commented Sep 30, 2022

fwyzard commented Sep 30, 2022

fwyzard commented Sep 30, 2022

cmsbuild commented Sep 30, 2022

fwyzard commented Sep 30, 2022

cmsbuild commented Sep 30, 2022

Comparison Summary

GPU Comparison Summary

cmsbuild commented Sep 30, 2022

fwyzard commented Sep 30, 2022

cmsbuild commented Oct 1, 2022

Unit Tests

Comparison Summary

GPU Comparison Summary

fwyzard commented Oct 3, 2022

fwyzard commented Oct 3, 2022

cmsbuild commented Oct 3, 2022

fwyzard commented Oct 3, 2022

cmsbuild commented Oct 3, 2022

fwyzard commented Oct 3, 2022

fwyzard commented Oct 3, 2022

cmsbuild commented Oct 4, 2022

Comparison Summary

GPU Comparison Summary

perrotta commented Oct 4, 2022

cmsbuild commented Sep 30, 2022 •

edited

Loading