Improve support for HIP/ROCm and Alpaka #40678

fwyzard · 2023-02-02T18:02:31Z

PR description:

Update CMS code for Alpaka devel as of 2023.02.01, that includes some functions and traits previously defined in CMSSW.

Add or improve support for HIP/ROCm in HeterogeneousCore tools and SoA DataFormats, including a dedicated unit test.

This PR requires

PR validation:

Old and new unit tests pass.

fwyzard · 2023-02-02T18:03:01Z

please test with cms-sw/cmssw-config#91, cms-sw/cmssw-config#91

cmsbuild · 2023-02-02T18:09:31Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40678/34028

ERROR: Build errors found during clang-tidy run.

                                                                           ^~~~~~~
                                                                           Queue
--
HeterogeneousCore/AlpakaInterface/interface/CachingAllocator.h:87:92: error: expected unqualified-id [clang-diagnostic-error]
            typename = std::enable_if_t<alpaka::isDevice<TDev> and alpaka::isQueue<TQueue>>>
                                                                                           ^
HeterogeneousCore/AlpakaInterface/interface/devices.h:15:71: error: no member named 'isPlatform' in namespace 'alpaka' [clang-diagnostic-error]
    template <typename TPlatform, typename = std::enable_if_t<alpaka::isPlatform<TPlatform>>>
                                                                      ^
HeterogeneousCore/AlpakaInterface/interface/devices.h:15:82: error: 'TPlatform' does not refer to a value [clang-diagnostic-error]
    template <typename TPlatform, typename = std::enable_if_t<alpaka::isPlatform<TPlatform>>>
                                                                                 ^
HeterogeneousCore/AlpakaInterface/interface/devices.h:15:24: note: declared here
--
HeterogeneousCore/AlpakaInterface/interface/devices.h:15:93: error: expected unqualified-id [clang-diagnostic-error]
    template <typename TPlatform, typename = std::enable_if_t<alpaka::isPlatform<TPlatform>>>
                                                                                            ^
HeterogeneousCore/AlpakaInterface/interface/devices.h:34:69: error: no member named 'isPlatform' in namespace 'alpaka' [clang-diagnostic-error]
  template <typename TPlatform, typename = std::enable_if_t<alpaka::isPlatform<TPlatform>>>
                                                                    ^
Suppressed 588 warnings (587 in non-user code, 1 with check filters).
--
gmake: *** [config/SCRAM/GMake/Makefile.coderules:129: code-checks] Error 2
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

fwyzard · 2023-02-03T05:37:55Z

code-checks with cms.week0.PR_ac9e8379/53.0-004e1992113b32c3ea5cd84a3366d985

fwyzard · 2023-02-03T05:47:31Z

please test with cms-sw/cmsdist#8284,cms-sw/cmsdist#8285

cmsbuild · 2023-02-03T05:54:16Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40678/34030

This PR adds an extra 52KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DataFormats/Portable/interface/PortableCollection.h modified in PR(s): Add support to multi layout portable collections (any number) #40285
- File DataFormats/Portable/interface/PortableDeviceCollection.h modified in PR(s): Add support to multi layout portable collections (any number) #40285
- File DataFormats/Portable/interface/PortableHostCollection.h modified in PR(s): Fix memory leaks in PortableHostCollection deserialization #40492, Add support to multi layout portable collections (any number) #40285
- File HeterogeneousCore/AlpakaTest/plugins/alpaka/TestAlgo.dev.cc modified in PR(s): Simplify Alpaka ESProducer model by making the data copies to device implicit #40403, Add support to multi layout portable collections (any number) #40285

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40678/34030/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40678/34030/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

fwyzard · 2023-02-03T11:13:45Z

please test with cms-sw/cmsdist#8284,cms-sw/cmsdist#8285

fwyzard · 2023-02-03T11:18:39Z

code-checks with cms.week0.PR_ac9e8379/53.0-004e1992113b32c3ea5cd84a3366d985

cmsbuild · 2023-02-03T11:24:57Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40678/34037

This PR adds an extra 52KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DataFormats/Portable/interface/PortableCollection.h modified in PR(s): Add support to multi layout portable collections (any number) #40285
- File DataFormats/Portable/interface/PortableDeviceCollection.h modified in PR(s): Add support to multi layout portable collections (any number) #40285
- File DataFormats/Portable/interface/PortableHostCollection.h modified in PR(s): Fix memory leaks in PortableHostCollection deserialization #40492, Add support to multi layout portable collections (any number) #40285
- File HeterogeneousCore/AlpakaTest/plugins/alpaka/TestAlgo.dev.cc modified in PR(s): Simplify Alpaka ESProducer model by making the data copies to device implicit #40403, Add support to multi layout portable collections (any number) #40285

fwyzard · 2023-02-03T15:52:41Z

code-checks with cms.week0.PR_ac9e8379/53.0-004e1992113b32c3ea5cd84a3366d985

fwyzard · 2023-02-03T15:57:34Z

please test with cms-sw/cmsdist#8284,cms-sw/cmsdist#8285

cmsbuild · 2023-02-03T16:05:05Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40678/34043

This PR adds an extra 20KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DataFormats/Portable/interface/PortableCollection.h modified in PR(s): Add support to multi layout portable collections (any number) #40285
- File DataFormats/Portable/interface/PortableDeviceCollection.h modified in PR(s): Add support to multi layout portable collections (any number) #40285
- File DataFormats/Portable/interface/PortableHostCollection.h modified in PR(s): Fix memory leaks in PortableHostCollection deserialization #40492, Add support to multi layout portable collections (any number) #40285
- File HeterogeneousCore/AlpakaTest/plugins/alpaka/TestAlgo.dev.cc modified in PR(s): Add support to multi layout portable collections (any number) #40285

cmsbuild · 2023-02-03T16:05:37Z

Pull request #40678 was updated. @makortel, @fwyzard can you please check and sign again.

fwyzard · 2023-02-03T16:41:33Z

All tests ran fine locally.

fwyzard · 2023-02-03T17:36:54Z

+1

cmsbuild · 2023-02-03T17:37:17Z

This pull request is fully signed and it will be integrated in one of the next master IBs after it passes the integration tests. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild · 2023-02-03T19:43:23Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d475d4/30387/summary.html
COMMIT: a14e8fc
CMSSW: CMSSW_13_0_X_2023-02-03-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/40678/30387/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d475d4/30387/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d475d4/30387/git-merge-result

Comparison Summary

Summary:

You potentially removed 15 lines from the logs
Reco comparison results: 80 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3555495
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3555470
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: no differences found

perrotta · 2023-02-05T18:01:15Z

+1

Differences in tests come from Code to process phase2 version of deepTauID v2p5 #40622

cmsbuild added this to the CMSSW_13_0_X milestone Feb 2, 2023

cmsbuild added pending-signatures tests-pending orp-pending code-checks-pending heterogeneous-pending labels Feb 2, 2023

cmsbuild added tests-started requires-external and removed tests-pending labels Feb 2, 2023

This was referenced Feb 2, 2023

Update Alpaka and add support for ROCm cms-sw/cmsdist#8284

Merged

Implement HIP/ROCm build rules and test packages #40627

Closed

cmsbuild added code-checks-rejected and removed code-checks-pending labels Feb 2, 2023

smuzaffar mentioned this pull request Feb 2, 2023

[BuildRules] Reorder Rocm flags so that those can override compilers flags cms-sw/cmsdist#8285

Merged

cmsbuild added code-checks-pending and removed code-checks-rejected labels Feb 3, 2023

cmsbuild added code-checks-rejected and removed code-checks-pending labels Feb 3, 2023

fwyzard force-pushed the ROCm_developments branch from 23d8b79 to 5dc0adc Compare February 3, 2023 11:13

cmsbuild added code-checks-pending and removed code-checks-rejected labels Feb 3, 2023

cmsbuild removed the code-checks-pending label Feb 3, 2023

cmsbuild added pending-signatures tests-pending code-checks-pending heterogeneous-pending and removed code-checks-approved heterogeneous-approved labels Feb 3, 2023

cmsbuild added tests-started requires-external and removed tests-pending labels Feb 3, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 3, 2023

cmsbuild added fully-signed heterogeneous-approved and removed pending-signatures heterogeneous-pending labels Feb 3, 2023

cmsbuild added tests-approved and removed tests-started labels Feb 3, 2023

cmsbuild added orp-approved and removed orp-pending labels Feb 5, 2023

cmsbuild merged commit 06f284e into cms-sw:master Feb 5, 2023

fwyzard deleted the ROCm_developments branch March 9, 2023 21:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve support for HIP/ROCm and Alpaka #40678

Improve support for HIP/ROCm and Alpaka #40678

fwyzard commented Feb 2, 2023

fwyzard commented Feb 2, 2023

cmsbuild commented Feb 2, 2023

fwyzard commented Feb 3, 2023

fwyzard commented Feb 3, 2023

cmsbuild commented Feb 3, 2023

fwyzard commented Feb 3, 2023

fwyzard commented Feb 3, 2023

cmsbuild commented Feb 3, 2023

fwyzard commented Feb 3, 2023

fwyzard commented Feb 3, 2023

cmsbuild commented Feb 3, 2023

cmsbuild commented Feb 3, 2023

fwyzard commented Feb 3, 2023

fwyzard commented Feb 3, 2023

cmsbuild commented Feb 3, 2023

cmsbuild commented Feb 3, 2023

perrotta commented Feb 5, 2023 •

edited

Loading

Improve support for HIP/ROCm and Alpaka #40678

Improve support for HIP/ROCm and Alpaka #40678

Conversation

fwyzard commented Feb 2, 2023

PR description:

PR validation:

fwyzard commented Feb 2, 2023

cmsbuild commented Feb 2, 2023

fwyzard commented Feb 3, 2023

fwyzard commented Feb 3, 2023

cmsbuild commented Feb 3, 2023

fwyzard commented Feb 3, 2023

fwyzard commented Feb 3, 2023

cmsbuild commented Feb 3, 2023

fwyzard commented Feb 3, 2023

fwyzard commented Feb 3, 2023

cmsbuild commented Feb 3, 2023

cmsbuild commented Feb 3, 2023

fwyzard commented Feb 3, 2023

fwyzard commented Feb 3, 2023

cmsbuild commented Feb 3, 2023

cmsbuild commented Feb 3, 2023

Comparison Summary

perrotta commented Feb 5, 2023 • edited Loading

perrotta commented Feb 5, 2023 •

edited

Loading