Patatrack integration - common tools updates (3/N) #31110

fwyzard · 2020-08-10T17:22:07Z

PR description:

Update tools for working on GPUs, following the Patatrack developments since #29110.

PR validation:

Changes in use in the Patatrack releases.

if this PR is a backport please specify the original PR and why you need to backport that PR:

Includes changes from:

Add options to cmsCudaRebuild.sh (Add various options to cmsCudaRebuild.sh cms-patatrack/cmssw#469)
Improve the CUDAMonitoringService (Improve CUDAMonitoringService cms-patatrack/cmssw#506)
Reduce GPU memory usage (use a dynamic buffer for CA cells components, adjust allocator growing factor to reduce memory used cms-patatrack/cmssw#509)
Update cmsCudaSetup.sh handling of CUDA_FLAGS (Update cmsCudaSetup.sh handling of CUDA_FLAGS cms-patatrack/cmssw#518)
Remove "cuda_cxx17.h" (Remove "cuda_cxx17.h" cms-patatrack/cmssw#519)
Add host-to-device async copy for host::noncached::unique_ptr (Add host-to-device async copy for host::noncached::unique_ptr cms-patatrack/cmssw#529)

Adjust the growth factor in the caching allocators to use more granular bins, reducing the memory wasted by the allocations.

CachingDeviceAllocator: - add the device allocator status to the public interface - monitor the requested amount of bytes in addition to the allocated amount CUDAMonitoringService: - print CUDA memory information after each module, including stats from caching allocator

Remove the C++ 17 compatibility functions now that CUDA 11.0 and later support C++ 17 natively.

Adds the type-safe wrappers for performing host-to-device asynchronous copies of data from a cms::cuda::host::noncached::unique_ptr<T> to a cms::cuda::device::unique_ptr<T>.

Add options to cmsCudaRebuild.sh: - "-v" to make scram be verbose and print all compilation commands - "-g" to enable host and device debug symbols - "-G" to enable host and device debug symbols, and device asserts

Support having multiple `-gencode arch=compute_..,code=sm_..` options on a single line, as well as one set of options on each line.

fwyzard · 2020-08-10T17:22:27Z

@makortel @VinInn FYI

fwyzard · 2020-08-10T17:22:35Z

@cmsbuild, please test

fwyzard · 2020-08-10T17:22:39Z

enable gpu

cmsbuild · 2020-08-10T17:24:25Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-08-10T17:28:35Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31110/17680

This PR adds an extra 20KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File HeterogeneousCore/CUDAServices/plugins/CUDAMonitoringService.cc modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File HeterogeneousCore/CUDAServices/scripts/cmsCudaRebuild.sh modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File HeterogeneousCore/CUDAServices/scripts/cmsCudaSetup.sh modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File HeterogeneousCore/CUDAUtilities/interface/copyAsync.h modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File HeterogeneousCore/CUDAUtilities/interface/cudaCompat.h modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File HeterogeneousCore/CUDAUtilities/interface/cuda_cxx17.h modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File HeterogeneousCore/CUDAUtilities/interface/deviceAllocatorStatus.h modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File HeterogeneousCore/CUDAUtilities/interface/prefixScan.h modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File HeterogeneousCore/CUDAUtilities/src/CachingDeviceAllocator.h modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File HeterogeneousCore/CUDAUtilities/src/deviceAllocatorStatus.cc modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File HeterogeneousCore/CUDAUtilities/src/getCachingDeviceAllocator.h modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983

cmsbuild · 2020-08-10T17:29:03Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-08-10-1100/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/8691/console Started: 2020/08/10 19:41

cmsbuild · 2020-08-10T17:29:04Z

A new Pull Request was created by @fwyzard (Andrea Bocci) for master.

It involves the following packages:

HeterogeneousCore/CUDAServices
HeterogeneousCore/CUDAUtilities

@makortel, @cmsbuild, @fwyzard can you please review it and eventually sign? Thanks.
@makortel, @rovere this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2020-08-10T19:19:11Z

+1
Tested at: 99ea961
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-bbab89/8691/summary.html
CMSSW: CMSSW_11_2_X_2020-08-10-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-08-10T19:19:14Z

Comparison job queued.

cmsbuild · 2020-08-10T20:47:37Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-bbab89/8691/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2612401
DQMHistoTests: Total failures: 5
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2612347
DQMHistoTests: Total skipped: 48
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 34 files compared)
DQMHistoSizes: changed ( 10224.0 ): -0.004 KiB MessageLogger/Warnings
Checked 149 log files, 22 edm output root files, 35 DQM output files

fwyzard · 2020-08-10T22:45:30Z

+1

cmsbuild · 2020-08-10T22:45:56Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2020-08-11T05:48:39Z

+1

VinInn and others added 7 commits August 10, 2020 19:01

Fix a possible data race in the prefix scan

d1ce91e

Reduce the GPU caching allocator memory usage

597ffeb

Adjust the growth factor in the caching allocators to use more granular bins, reducing the memory wasted by the allocations.

Remove "cuda_cxx17.h"

79f56c6

Remove the C++ 17 compatibility functions now that CUDA 11.0 and later support C++ 17 natively.

Add host-to-device async copy for host::noncached::unique_ptr

4eafc1c

Adds the type-safe wrappers for performing host-to-device asynchronous copies of data from a cms::cuda::host::noncached::unique_ptr<T> to a cms::cuda::device::unique_ptr<T>.

Add options to cmsCudaRebuild.sh

250efee

Add options to cmsCudaRebuild.sh: - "-v" to make scram be verbose and print all compilation commands - "-g" to enable host and device debug symbols - "-G" to enable host and device debug symbols, and device asserts

Update cmsCudaSetup.sh handling of CUDA_FLAGS

99ea961

Support having multiple `-gencode arch=compute_..,code=sm_..` options on a single line, as well as one set of options on each line.

cmsbuild added this to the CMSSW_11_2_X milestone Aug 10, 2020

cmsbuild added code-checks-pending comparison-pending heterogeneous-pending orp-pending pending-signatures tests-pending labels Aug 10, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 10, 2020

cmsbuild added tests-started and removed tests-pending labels Aug 10, 2020

cmsbuild added tests-approved and removed tests-started labels Aug 10, 2020

cmsbuild added comparison-available and removed comparison-pending labels Aug 10, 2020

cmsbuild added fully-signed heterogeneous-approved and removed heterogeneous-pending pending-signatures labels Aug 10, 2020

cmsbuild added orp-approved and removed orp-pending labels Aug 11, 2020

cmsbuild merged commit 8fcaead into cms-sw:master Aug 11, 2020

fwyzard deleted the patatrack_integration_3_N_updates branch October 5, 2020 16:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Patatrack integration - common tools updates (3/N) #31110

Patatrack integration - common tools updates (3/N) #31110

fwyzard commented Aug 10, 2020

fwyzard commented Aug 10, 2020

fwyzard commented Aug 10, 2020

fwyzard commented Aug 10, 2020

cmsbuild commented Aug 10, 2020

cmsbuild commented Aug 10, 2020

cmsbuild commented Aug 10, 2020 •

edited

Loading

cmsbuild commented Aug 10, 2020

cmsbuild commented Aug 10, 2020

cmsbuild commented Aug 10, 2020

cmsbuild commented Aug 10, 2020

fwyzard commented Aug 10, 2020

cmsbuild commented Aug 10, 2020

qliphy commented Aug 11, 2020

Patatrack integration - common tools updates (3/N) #31110

Patatrack integration - common tools updates (3/N) #31110

Conversation

fwyzard commented Aug 10, 2020

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

fwyzard commented Aug 10, 2020

fwyzard commented Aug 10, 2020

fwyzard commented Aug 10, 2020

cmsbuild commented Aug 10, 2020

cmsbuild commented Aug 10, 2020

cmsbuild commented Aug 10, 2020 • edited Loading

cmsbuild commented Aug 10, 2020

cmsbuild commented Aug 10, 2020

cmsbuild commented Aug 10, 2020

cmsbuild commented Aug 10, 2020

fwyzard commented Aug 10, 2020

cmsbuild commented Aug 10, 2020

qliphy commented Aug 11, 2020

cmsbuild commented Aug 10, 2020 •

edited

Loading