Reimplement the PortableCollection and related tests in CUDA [12.5.x] #39321
Conversation
|
A new Pull Request was created by @fwyzard (Andrea Bocci) for CMSSW_12_5_X. It involves the following packages:
The following packages do not have a category, yet: CUDADataFormats/PortableTestObjects @cmsbuild, @makortel, @fwyzard can you please review it and eventually sign? Thanks. cms-bot commands are listed here |
fwyzard
force-pushed
the
cuda_backport_alpaka_tests
branch
from
30a5539
to
c1f6600
Compare
fwyzard
changed the title
|
backport #39319 |
|
enable gpu |
|
please test |
cmsbuild
added
backport
tests-started
tests-rejected
and removed
tests-pending
tests-started
labels
|
-1 Failed Tests: Build ClangBuild BuildI found compilation error when building: /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_5_X_2022-09-06-1100/src/HeterogeneousCore/AlpakaInterface/interface/CachedBufAlloc.h:76:42: required from 'static alpaka::BufCpu cms::alpakatools::traits::CachedBufAlloc, void>::allocCachedBuf(const alpaka::DevCpu&, alpaka::QueueCudaRtNonBlocking, const TExtent&) [with TExtent = alpaka::Vec, unsigned int>; TElem = std::byte; TDim = std::integral_constant; TIdx = unsigned int; alpaka::QueueCudaRtNonBlocking = alpaka::uniform_cuda_hip::detail::QueueUniformCudaHipRt]'
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_5_X_2022-09-06-1100/src/HeterogeneousCore/AlpakaInterface/interface/CachedBufAlloc.h:202:99: required from 'auto cms::alpakatools::allocCachedBuf(const TDev&, TQueue, const TExtent&) [with TElem = std::byte; TIdx = unsigned int; TExtent = alpaka::Vec, unsigned int>; TQueue = alpaka::uniform_cuda_hip::detail::QueueUniformCudaHipRt; TDev = alpaka::DevCpu; = void]'
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_5_X_2022-09-06-1100/src/HeterogeneousCore/AlpakaInterface/interface/memory.h:117:58: required from 'std::enable_if_t<((is_queue_v && is_unbounded_array_v) && (! is_array_v::type>)), typename cms::alpakatools::detail::buffer_type::type> cms::alpakatools::make_host_buffer(const TQueue&, alpaka_common::Extent) [with T = std::byte []; TQueue = alpaka::uniform_cuda_hip::detail::QueueUniformCudaHipRt; std::enable_if_t<((is_queue_v && is_unbounded_array_v) && (! is_array_v::type>)), typename cms::alpakatools::detail::buffer_type::type> = alpaka::BufCpu, unsigned int>; typename cms::alpakatools::detail::buffer_type::type = alpaka::BufCpu, unsigned int>; typename std::remove_extent<_Tp>::type = std::byte; alpaka_common::Extent = unsigned int]'
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_5_X_2022-09-06-1100/src/DataFormats/Portable/interface/PortableHostCollection.h:36:64: required from 'PortableHostCollection::PortableHostCollection(int32_t, const TQueue&) [with TQueue = alpaka::uniform_cuda_hip::detail::QueueUniformCudaHipRt; = void; T = portabletest::TestSoALayout<>; int32_t = int]'
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_5_X_2022-09-06-1100/src/HeterogeneousCore/AlpakaTest/plugins/alpaka/TestAlpakaTranscriber.cc:43:100: required from here
/cvmfs/cms-ib.cern.ch/nweek-02749/el8_amd64_gcc10/external/alpaka/develop-20220811-f73576a1328c2c84ceb84f49e8371846/include/alpaka/mem/buf/Traits.hpp:140:86: error: incomplete type 'alpaka::trait::BufAllocMapped, std::integral_constant, std::byte, long unsigned int>' used in nested name specifier
140 | return trait::BufAllocMapped, TIdx, TDev>::allocMappedBuf(host, dev, extent);
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~
gmake: *** [tmp/el8_amd64_gcc10/src/HeterogeneousCore/AlpakaTest/plugins/HeterogeneousCoreAlpakaTestPluginsPortableCudaAsync/alpaka/TestAlpakaTranscriber.cc.o] Error 1
>> Cuda Device Link tmp/el8_amd64_gcc10/src/HeterogeneousCore/AlpakaTest/plugins/HeterogeneousCoreAlpakaTestPluginsPortableCudaAsync/HeterogeneousCoreAlpakaTestPluginsPortableCudaAsync_cudadlink.o
>> Building alpaka/cuda edm plugin tmp/el8_amd64_gcc10/src/HeterogeneousCore/AlpakaTest/plugins/HeterogeneousCoreAlpakaTestPluginsPortableCudaAsync/libHeterogeneousCoreAlpakaTestPluginsPortableCudaAsync.so
Clang BuildI found compilation error while trying to compile with clang. Command used: >> Entering Package DataFormats/SoATemplate >> Entering Package HeterogeneousCore/AlpakaTest >> Entering Package HeterogeneousCore/CUDATest >> Entering Package HeterogeneousCore/CUDAUtilities >> Compile sequence completed for CMSSW CMSSW_12_5_X_2022-09-06-1100 gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 1 + eval scram build outputlog '&&' '(python3' /data/cmsbld/jenkins/workspace/ib-run-pr-tests/cms-bot/buildLogAnalyzer.py --logDir /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_5_X_2022-09-06-1100/tmp/el8_amd64_gcc10/cache/log/src '||' 'true)' ++ scram build outputlog >> Entering Package CUDADataFormats/Common Entering library rule at CUDADataFormats/Common >> Compiling /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_5_X_2022-09-06-1100/src/CUDADataFormats/Common/src/ProductBase.cc |
|
please test with cms-sw/cmsdist#8069 |
fwyzard
force-pushed
the
cuda_backport_alpaka_tests
branch
from
c1f6600
to
e84b4e3
Compare
|
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8ecf42/27409/summary.html GPU Comparison SummarySummary:
Comparison SummarySummary:
|
Move the test logic to a bash script to simplify the BuildFile.xml.
fwyzard
force-pushed
the
cuda_backport_alpaka_tests
branch
from
c9a50cb
to
d2e962a
Compare
|
please test |
|
+heterogeneous |
cmsbuild
added
fully-signed
heterogeneous-approved
and removed
pending-signatures
heterogeneous-pending
labels
|
This pull request is fully signed and it will be integrated in one of the next CMSSW_12_5_X IBs after it passes the integration tests and once validation in the development release cycle CMSSW_12_6_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2) |
|
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8ecf42/27418/summary.html GPU Comparison SummarySummary:
Comparison SummarySummary:
|
|
+1 |
PR description:
To ease the migration from CUDA to Alpaka, reimplement in CUDA the
PortableHostCollectionandPortableDeviceCollectionoriginally developed for the Alpaka data formats, along with thePortableTestObjectstests.The CUDA version is in the
cms::cudanamespace.The data format functionality requires extending the
cms::cuda::host::make_unique<T>to support pageable (i.e. non-pinned) host memory.PR validation:
The new unit tests run successfully, both with and without GPUs:
Regarding the changes to the
cms::cuda::host::unique_ptr<T>, they were tested also in CMSSW 12.4.8 on a GPU-enabled HLT workflow, to make sure they do not have any negative impact on the performance.Vanilla CMSSW_12_4_8
With the equivalent changes as in this PR:
If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:
Backport of #39319 to CMSSW 12.5.x.