[BuildRules] Enable alpaka ROCm backend #8346

fwyzard · 2023-03-02T11:08:01Z

Same as #8301 with a clean commit history.

fwyzard · 2023-03-02T11:08:07Z

please test

cmsbuild · 2023-03-02T11:08:20Z

A new Pull Request was created by @fwyzard (Andrea Bocci) for branch IB/CMSSW_13_1_X/master.

@smuzaffar, @aandvalenzuela, @iarspider can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.
cms-bot commands are listed here

fwyzard · 2023-03-02T11:08:23Z

please test for el8_ppc64le_gcc11

fwyzard · 2023-03-02T11:26:20Z

urgent

backport needed for 13.0.0

smuzaffar · 2023-03-02T12:59:38Z

test parameters:

addpkg = FWCore,HeterogeneousCore, DataFormats/Portable, HeterogeneousTest, DataFormats/PortableTestObjects

smuzaffar · 2023-03-02T13:01:00Z

please test

lets make sure that we rebuild packages with rocm dict

cmsbuild · 2023-03-02T13:11:32Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3590a5/30991/summary.html
COMMIT: a35ac47
CMSSW: CMSSW_13_1_X_2023-03-01-2300/el8_ppc64le_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8346/30991/install.sh to create a dev area with all the needed externals and cmssw changes.

smuzaffar · 2023-03-02T13:14:20Z

please test for el8_ppc64le_gcc11

cmsbuild · 2023-03-02T13:49:32Z

-1

Failed Tests: Build
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3590a5/30999/summary.html
COMMIT: a35ac47
CMSSW: CMSSW_13_1_X_2023-03-02-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8346/30999/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@cms-sw [SimG4CMS-Calo] Update HFShowerLibrary_run3_v7 #8292

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3590a5/30999/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3590a5/30999/git-merge-result

Build

I found compilation error when building:

>> Cuda Device Link tmp/el8_amd64_gcc11/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestKernelCudaAsync/alpakaTestKernelCudaAsync_cudadlink.o 
>> Building alpaka/cuda binary alpakaTestKernelCudaAsync
Copying tmp/el8_amd64_gcc11/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestKernelCudaAsync/alpakaTestKernelCudaAsync to productstore area:
>> Building alpaka/rocm binary alpakaTestKernelROCmAsync
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc11/external/gcc/11.2.1-f9b9dfdd886f71cd63f5538223d8f161/bin/../lib/gcc/x86_64-redhat-linux-gnu/11.2.1/../../../../x86_64-redhat-linux-gnu/bin/ld: cannot find tmp/el8_amd64_gcc11/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestKernelROCmAsync/alpaka/testKernel.dev.cc.o: No such file or directory
collect2: error: ld returned 1 exit status
>> Deleted: tmp/el8_amd64_gcc11/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestKernelROCmAsync/alpakaTestKernelROCmAsync
gmake: *** [tmp/el8_amd64_gcc11/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestKernelROCmAsync/alpakaTestKernelROCmAsync] Error 1
>> Compiling alpaka/serial /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_13_1_X_2023-03-02-1100/src/HeterogeneousCore/AlpakaInterface/test/alpaka/testKernel.dev.cc
>> Building alpaka/serial binary alpakaTestKernelSerialSync
Copying tmp/el8_amd64_gcc11/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestKernelSerialSync/alpakaTestKernelSerialSync to productstore area:

smuzaffar · 2023-03-02T16:12:37Z

please test for el8_ppc64le_gcc11

smuzaffar · 2023-03-02T16:12:50Z

please test

cmsbuild · 2023-03-02T16:12:50Z

Pull request #8346 was updated.

cmsbuild · 2023-03-02T16:47:55Z

-1

Failed Tests: Build
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3590a5/31008/summary.html
COMMIT: 9817990
CMSSW: CMSSW_13_1_X_2023-03-01-2300/el8_ppc64le_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8346/31008/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@cms-sw [SimG4CMS-Calo] Update HFShowerLibrary_run3_v7 #8292

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3590a5/31008/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3590a5/31008/git-merge-result

Build

I found compilation error when building:

Entering library rule at HeterogeneousCore/SonicTriton
>> Compiling  /scratch/cmsbuild/jenkins_b/workspace/ib-run-pr-tests/CMSSW_13_1_X_2023-03-01-2300/src/HeterogeneousCore/SonicTriton/src/TritonClient.cc
>> Compiling  /scratch/cmsbuild/jenkins_b/workspace/ib-run-pr-tests/CMSSW_13_1_X_2023-03-01-2300/src/HeterogeneousCore/SonicTriton/src/TritonData.cc
>> Compiling  /scratch/cmsbuild/jenkins_b/workspace/ib-run-pr-tests/CMSSW_13_1_X_2023-03-01-2300/src/HeterogeneousCore/SonicTriton/src/TritonException.cc
>> Compiling  /scratch/cmsbuild/jenkins_b/workspace/ib-run-pr-tests/CMSSW_13_1_X_2023-03-01-2300/src/HeterogeneousCore/SonicTriton/src/TritonMemResource.cc
/scratch/cmsbuild/jenkins_b/workspace/ib-run-pr-tests/CMSSW_13_1_X_2023-03-01-2300/src/HeterogeneousCore/SonicTriton/src/TritonData.cc:7:10: fatal error: triton/common/model_config.h: No such file or directory
    7 | #include "triton/common/model_config.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
>> Compiling  /scratch/cmsbuild/jenkins_b/workspace/ib-run-pr-tests/CMSSW_13_1_X_2023-03-01-2300/src/HeterogeneousCore/SonicTriton/src/TritonService.cc
/scratch/cmsbuild/jenkins_b/workspace/ib-run-pr-tests/CMSSW_13_1_X_2023-03-01-2300/src/HeterogeneousCore/SonicTriton/src/TritonData.cc:7:10: fatal error: triton/common/model_config.h: No such file or directory

cmsbuild · 2023-03-02T17:03:52Z

-1

Failed Tests: Build
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3590a5/31007/summary.html
COMMIT: 9817990
CMSSW: CMSSW_13_1_X_2023-03-02-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8346/31007/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@cms-sw [SimG4CMS-Calo] Update HFShowerLibrary_run3_v7 #8292

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3590a5/31007/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3590a5/31007/git-merge-result

Build

I found compilation error when building:

TritonService
--- Registered EDM Plugin: HeterogenousCoreSonicTriton_plugins
Entering library rule at HeterogeneousCore/SonicTriton
>> Compiling  /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_13_1_X_2023-03-02-1100/src/HeterogeneousCore/SonicTriton/src/TritonClient.cc
>> Compiling  /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_13_1_X_2023-03-02-1100/src/HeterogeneousCore/SonicTriton/src/TritonData.cc
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_13_1_X_2023-03-02-1100/src/HeterogeneousCore/SonicTriton/src/TritonData.cc:7:10: fatal error: triton/common/model_config.h: No such file or directory
    7 | #include "triton/common/model_config.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_13_1_X_2023-03-02-1100/src/HeterogeneousCore/SonicTriton/src/TritonData.cc:7:10: fatal error: triton/common/model_config.h: No such file or directory
    7 | #include "triton/common/model_config.h"

fwyzard · 2023-03-02T17:10:01Z

Thanks, I really needed this to break the IBs even more.

… are dev.cc plugins sometimes have `*.cc` files and when scram tries to match these against device files `dev.cc` then it does not match so scram does not enable `rocm` rules. New build rules expand `*.cc` and then search for `dev.cc` files

smuzaffar · 2023-03-02T21:57:04Z

please test

cmsbuild · 2023-03-02T21:57:11Z

Pull request #8346 was updated.

smuzaffar · 2023-03-02T21:57:29Z

please test for el8_ppc64le_gcc11

fwyzard · 2023-03-02T23:05:33Z

@smuzaffar thanks for the update, V07-07-03 worked locally for me

fwyzard · 2023-03-03T00:18:56Z

---> test testFWCoreUtilities had ERRORS (for el8_ppc64le_gcc11) should be unrelated to these changes.
In fact, it's already present in the latest IB.

cmsbuild · 2023-03-03T00:21:11Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3590a5/31016/summary.html
COMMIT: 90bf522
CMSSW: CMSSW_13_1_X_2023-03-01-2300/el8_ppc64le_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8346/31016/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3590a5/31016/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3590a5/31016/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test testFWCoreUtilities had ERRORS

cmsbuild · 2023-03-03T01:26:22Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3590a5/31015/summary.html
COMMIT: 90bf522
CMSSW: CMSSW_13_1_X_2023-03-02-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8346/31015/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3590a5/31015/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3590a5/31015/git-merge-result

Comparison Summary

Summary:

You potentially added 11 lines to the logs
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3529699
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3529676
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: no differences found

smuzaffar · 2023-03-03T07:22:07Z

+externals

cmsbuild · 2023-03-03T07:22:32Z

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_13_1_X/master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

fwyzard · 2023-03-03T08:14:20Z

---> test testFWCoreUtilities had ERRORS (for el8_ppc64le_gcc11) should be unrelated to these changes. In fact, it's already present in the latest IB.

Traced to a bug in TBB, see cms-sw/cmssw#34647 (comment) .

[BuildRules] Enable alpaka ROCm backend

a35ac47

cmsbuild added externals-pending pending-signatures tests-started orp-pending labels Mar 2, 2023

This was referenced Mar 2, 2023

[BuildRules] Enable Alpaka/Rocm backend #8301

Closed

[BuildRules] Enable alpaka ROCm backend [13.0.x] #8347

Merged

cmsbuild added the urgent label Mar 2, 2023

cmsbuild added tests-rejected and removed tests-started labels Mar 2, 2023

fix for rocm dev.cc

9817990

cmsbuild added tests-started and removed tests-rejected labels Mar 2, 2023

cmsbuild added tests-rejected and removed tests-started labels Mar 2, 2023

cmsbuild added tests-started and removed tests-rejected labels Mar 2, 2023

cmsbuild added tests-approved and removed tests-started labels Mar 3, 2023

smuzaffar merged commit 744a8b8 into cms-sw:IB/CMSSW_13_1_X/master Mar 3, 2023

cmsbuild added externals-approved fully-signed and removed externals-pending pending-signatures labels Mar 3, 2023

fwyzard deleted the IB/CMSSW_13_1_X/master_enable_ROCm_backend branch March 3, 2023 07:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BuildRules] Enable alpaka ROCm backend #8346

[BuildRules] Enable alpaka ROCm backend #8346

fwyzard commented Mar 2, 2023

fwyzard commented Mar 2, 2023

cmsbuild commented Mar 2, 2023

fwyzard commented Mar 2, 2023

fwyzard commented Mar 2, 2023

smuzaffar commented Mar 2, 2023

smuzaffar commented Mar 2, 2023

cmsbuild commented Mar 2, 2023

smuzaffar commented Mar 2, 2023

cmsbuild commented Mar 2, 2023

smuzaffar commented Mar 2, 2023

smuzaffar commented Mar 2, 2023

cmsbuild commented Mar 2, 2023

cmsbuild commented Mar 2, 2023

cmsbuild commented Mar 2, 2023

fwyzard commented Mar 2, 2023

smuzaffar commented Mar 2, 2023

cmsbuild commented Mar 2, 2023

smuzaffar commented Mar 2, 2023

fwyzard commented Mar 2, 2023

fwyzard commented Mar 3, 2023

cmsbuild commented Mar 3, 2023

cmsbuild commented Mar 3, 2023

smuzaffar commented Mar 3, 2023

cmsbuild commented Mar 3, 2023

fwyzard commented Mar 3, 2023

[BuildRules] Enable alpaka ROCm backend #8346

[BuildRules] Enable alpaka ROCm backend #8346

Conversation

fwyzard commented Mar 2, 2023

fwyzard commented Mar 2, 2023

cmsbuild commented Mar 2, 2023

fwyzard commented Mar 2, 2023

fwyzard commented Mar 2, 2023

smuzaffar commented Mar 2, 2023

smuzaffar commented Mar 2, 2023

cmsbuild commented Mar 2, 2023

smuzaffar commented Mar 2, 2023

cmsbuild commented Mar 2, 2023

Build

smuzaffar commented Mar 2, 2023

smuzaffar commented Mar 2, 2023

cmsbuild commented Mar 2, 2023

cmsbuild commented Mar 2, 2023

Build

cmsbuild commented Mar 2, 2023

Build

fwyzard commented Mar 2, 2023

smuzaffar commented Mar 2, 2023

cmsbuild commented Mar 2, 2023

smuzaffar commented Mar 2, 2023

fwyzard commented Mar 2, 2023

fwyzard commented Mar 3, 2023

cmsbuild commented Mar 3, 2023

Unit Tests

cmsbuild commented Mar 3, 2023

Comparison Summary

smuzaffar commented Mar 3, 2023

cmsbuild commented Mar 3, 2023

fwyzard commented Mar 3, 2023