Patatrack integration - Pixel local reconstruction (9/N)

[fwyzard]

PR description:

Data formats and algorithms for the Pixel local reconstruction running on GPU.
Includes the Pixel unpacking, clustering, up to the rechits.
Includes the relevant changes to the Pixel validation sequences.

Some documentation about the pixel local reconstruction implementation in Patatrack can be found in these presentations:

• 3 September 2019: Patatrack Meeting, A fully Heterogeneous Pixel Reconstruction
• 12 December 2020: Patatrack Tracker/Tracking Knowledge Transfer, Patatrack Local Reconstruction
• 7 May 2020, Tracker DPG Workshop, A fully Heterogeneous Pixel Reconstruction

#32483 lists open issues to be followed up after the integration of this PR.

PR validation:

Changes in use in the Patatrack releases.

if this PR is a backport please specify the original PR and why you need to backport that PR:

Includes changes from:

• R2D: use GPU::SimpleVector for the error unpacking cms-patatrack/cmssw#14 R2D: use GPU::SimpleVector for the error unpacking
• GPU2CPU for clusters and RecHIts cms-patatrack/cmssw#18 GPU2CPU for clusters and RecHIts
• Add workflows for Riemann fit and GPU cms-patatrack/cmssw#20 Add workflows for Riemann fit and GPU
• Remove debug messages cms-patatrack/cmssw#21 Remove debug messages
• Remove -O2/-O3 from CUDA flags cms-patatrack/cmssw#29 Remove -O2/-O3 from CUDA flags
• Use the gpu modifier to read the pixel clusters from the unpacker cms-patatrack/cmssw#31 Use the gpu modifier to read the pixel clusters from the unpacker
• Remove the use of #pragma once cms-patatrack/cmssw#47 Remove the use of #pragma once
• Clean up some GPU- and non-GPU-related pixel tracking code cms-patatrack/cmssw#49 Clean up some GPU- and non-GPU-related pixel tracking code
• Heterogeneous version of Raw2Cluster and RecHit cms-patatrack/cmssw#62 Implement a Heterogeneous version of Raw2Cluster and RecHit
• faster cluster, now does not requires to know number of modules cms-patatrack/cmssw#68 Faster clustering, now does not requires to know number of modules
• Prototype for EventSetup data on GPUs cms-patatrack/cmssw#77 Prototype for EventSetup data on GPUs
• GPU: better hits cms-patatrack/cmssw#81 Migrated PixelRecHit to Heterogeneous producer
• Various fixes and cleanup cms-patatrack/cmssw#87 Various fixes and cleanup
• Next prototype of the framework integration cms-patatrack/cmssw#100 Next prototype of the framework integration
• Various fixes and cleanup cms-patatrack/cmssw#101 Various fixes and cleanup
• Fix synchronisation problems in the clusterizer cms-patatrack/cmssw#102 Fix synchronisation problems in the clusterizer
• Cluster2TP assoc on GPU cms-patatrack/cmssw#105 Heterogeneous ClusterTPAssociation
• Remove all remaining calls to cudaStreamSynchronize() cms-patatrack/cmssw#109 Remove all remaining calls to cudaStreamSynchronize()
• Make all device->host cudaMemcpyAsync's to use pinned memory cms-patatrack/cmssw#112 Make all device->host cudaMemcpyAsync transfers use pinned memory
• Fix memory initialisation problems in the clusterizer (again) cms-patatrack/cmssw#114 Fix memory initialisation problems in the clusterizer (again)
• Add optional flags to disable SOA->legacy conversion and GPU->CPU transfer cms-patatrack/cmssw#132 Add optional flags to disable SOA->legacy conversion and GPU->CPU transfer
• Cleanup after merging with CMSSW 10.2.2 cms-patatrack/cmssw#134 Cleanup after merging with CMSSW 10.2.2
• Doublet tuning and new vertex algo cms-patatrack/cmssw#158 Tune and speed up doublet algo
• Introduce Cluster Charge Cut, optimize Histogram (bucket sorting) cms-patatrack/cmssw#171 Introduce Cluster Charge Cut, optimize Histogram (bucket sorting)
• Add infrastructure around cub CachingDeviceAllocator, and use it in SiPixelRawToCluster cms-patatrack/cmssw#172 Add infrastructure around cub CachingDeviceAllocator, and use it in SiPixelRawToCluster
• Work around atomicAdd problem on Volta cms-patatrack/cmssw#180 Work around atomicAdd synchronisation problem on Volta
• Suppress asserts in the GPU code, unless GPU_DEBUG is defined cms-patatrack/cmssw#186 Suppress asserts in the GPU code, unless GPU_DEBUG is defined
• Migrate tracker local reconstruction and pixel tracking to Tasks (backport #25163) cms-patatrack/cmssw#202 Migrate tracker local reconstruction and pixel tracking to Tasks (backport Migrate tracker local reconstruction and pixel tracking to Tasks #25163)
• Address code style and quality issues cms-patatrack/cmssw#203 Address code style and quality issues
• Require allocated type to have only a trivial constructor for make_device_unique and make_host_unique cms-patatrack/cmssw#204 Require allocated type to have only a trivial constructor for make_device_unique and make_host_unique
• Fix access to uninitialised memory in RawToDigi_kernel cms-patatrack/cmssw#206 Fix access to uninitialised memory in RawToDigi_kernel
• faster track fit and faster clustering cms-patatrack/cmssw#216 Full workflow from raw data to pixel tracks and vertices on GPUs
• Fix setting the data pointer of error SimpleVector cms-patatrack/cmssw#236 Set the data pointer of error's SimpleVector
• Limit the pixel clusteriser to the nearest-neighbours cms-patatrack/cmssw#241 Limit the pixel clusteriser to nearest neighbours
• Fix invalid narrowing conversion from "unsigned int" to "unsigned char" cms-patatrack/cmssw#250 Fix invalid narrowing conversion from "unsigned int" to "unsigned char"
• Skip CUDA-related tests if no GPU is present cms-patatrack/cmssw#252 Skip CUDA-related tests if no GPU is present
• Fix or disable failing unit tests cms-patatrack/cmssw#253 Fix or disable failing unit tests
• Fix clang warnings cms-patatrack/cmssw#255 Fix warnings reported by clang
• Various updates to pixel track/vertex DQM and MTV cms-patatrack/cmssw#285 Various updates to pixel track/vertex DQM and MTV
• make gpu-cpu cluster matching deterministic cms-patatrack/cmssw#294 Make GPU-CPU cluster matching deterministic
• Apply code checks cms-patatrack/cmssw#315 Apply code checks
• Move BeamSpot transfer to GPU to its own producer cms-patatrack/cmssw#318 Move BeamSpot transfer to GPU to its own producer
• Tracking developments for review and merging cms-patatrack/cmssw#338 Clean up by clang-format
• Tracking developments for review and merging cms-patatrack/cmssw#338 Improve pixel doublets and CA, and extend debugging functionality
• Tracking developments for review and merging cms-patatrack/cmssw#338 Make SiPixelGainCalibrationForHLTGPU available on the CPU
• Tracking developments for review and merging cms-patatrack/cmssw#338 Migrate the pixel rechits producer and CA to the new heterogeneous framework
• Tracking developments for review and merging cms-patatrack/cmssw#338 Rework the GPU pixel track clusterizer and vertex finder
• Migrate ClusterTPAssociationHeterogeneous to the new framework  cms-patatrack/cmssw#346 Migrate ClusterTPAssociationHeterogeneous to the new framework
• Move to View cms-patatrack/cmssw#354 Migrate gpuPixelRecHits::getHits() kernel to use a View instead of multiple pointers
• Reorganize CUDAScopedContext cms-patatrack/cmssw#355 Reorganize CUDAScopedContext
• Move event and stream caches, and caching allocators out from CUDAService cms-patatrack/cmssw#364 Move event and stream caches, and caching allocators out from CUDAService
• Implement triplets in the pixel ntuplet producer cms-patatrack/cmssw#382 Implement triplets in the pixel ntuplet producer
• Port the whole pixel workflow to new heterogeneous framework cms-patatrack/cmssw#384 Port the whole pixel workflow to new heterogeneous framework
• Implement full Pixel SoA workflow on CPU cms-patatrack/cmssw#385 Implement full Pixel SoA workflow on CPU
• Fix clang warnings cms-patatrack/cmssw#387 Fix clang warnings
• Replace use of API wrapper stream and event with plain CUDA, part 1 cms-patatrack/cmssw#389 Replace use of API wrapper stream and event with plain CUDA, part 1
• Implement library-only wrappers for launching CUDA kernels cms-patatrack/cmssw#390 Implement library-only wrappers for launching CUDA kernels
• Replace CUDA API wrapper memory operations with native CUDA calls cms-patatrack/cmssw#395 Replace CUDA API wrapper memory operations with native CUDA calls
• Replace use of CUDA API wrapper unique_ptrs with CUDAUtilities unique_ptrs cms-patatrack/cmssw#396 Replace use of CUDA API wrapper unique_ptrs with CUDAUtilities unique_ptrs
• Use non-blocking CUDA streams cms-patatrack/cmssw#405 Use non-blocking CUDA streams
• Replace remaining cuda::device operations with native CUDA calls. cms-patatrack/cmssw#408 Replace cuda::device operations with native CUDA calls
• Drop obsolete heterogenous framework cms-patatrack/cmssw#416 Drop obsolete heterogenous framework
• Clean up the last remaining references to CUDA API Wrappers cms-patatrack/cmssw#417 Remove last references to CUDA API Wrappers
• move warning in producer cms-patatrack/cmssw#422 Move LogWarning into the EDProducer
• Rename exitSansCUDADevices to requireCUDADevices cms-patatrack/cmssw#423 Rename exitSansCUDADevices to requireCUDADevices
• [WIP] Follow-ups to the CUDA framework PR review cms-patatrack/cmssw#429 Implement changes from the CUDA framework review
• protect for too many pixels cms-patatrack/cmssw#436 Protect against too many pixels
• Update lumi ALCARECO configurations for heterogeneous pixel digi and cluster configuration cms-patatrack/cmssw#437 Update lumi ALCARECO configurations for heterogeneous pixel digi and cluster configuration
• avoid assert for phase2 cms-patatrack/cmssw#438 Work around: avoid assert in Phase 2 workflows
• Integrate the comments from the upstream PRs cms-patatrack/cmssw#442 Integrate the comments from the upstream PRs
• Apply code formatting fixes from upstream integration cms-patatrack/cmssw#445 Apply code formatting fixes from upstream integration
• Replace cub prefix-scan with home-brewed one. cms-patatrack/cmssw#447 Replace cub prefix scan with home-brewed one
• Remove dependency on cub cms-patatrack/cmssw#449 Remove dependency on cub
• Move CUDAHostAllocator to the cms::cuda namespace cms-patatrack/cmssw#464 Rename CUDAHostAllocator to cms::cuda::HostAllocator
• Use siPixelDigis.cpu instead of siPixelDigis cms-patatrack/cmssw#467 Use siPixelDigis.cpu instead of siPixelDigis
• Update Pixel gain calibration scheme (for Run3) cms-patatrack/cmssw#492 Update Pixel gain calibration scheme (for Run3)
• Add truncation to pixel charge on GPU cms-patatrack/cmssw#501 Add truncation to pixel charge on GPU
• Remove "cuda_cxx17.h" cms-patatrack/cmssw#519 Remove "cuda_cxx17.h"
• Remove duplicate definition of globalValidationHCALOnly cms-patatrack/cmssw#525 Remove duplicate definition of globalValidationHCALOnly
• Remove use of boost::mpl::vector for dependent records cms-patatrack/cmssw#527 Remove use of boost::mpl::vector for dependent records
• Remove misleading calls to std::move cms-patatrack/cmssw#546 Remove misleading calls to std::move
• Updating validation sequence for pixel only tracking workflows cms-patatrack/cmssw#548 Update the validation sequence for pixel-only tracking workflows
• Fix BS naming in siPixelRecHitsCUDAPreSplitting cms-patatrack/cmssw#551 Fix BS naming in siPixelRecHitsCUDAPreSplitting
• Update ESProducers following cms-sw#31556 cms-patatrack/cmssw#555 Update ESProducers following Remove setConsumes() from ESConsumesCollector #31556
• Define CUDA-specific attributes for compatiility with GCC cms-patatrack/cmssw#559 Define CUDA-specific attributes for compatility with GCC
• Bugfix: add missing GPU memory free to PixelCPEFast cms-patatrack/cmssw#570 Bugfix: add missing GPU memory free to PixelCPEFast
• Address some pixel local reco PR review comments cms-patatrack/cmssw#575 Address pixel local reco PR review comments
• fix esConsumes for PR #31721 cms-patatrack/cmssw#577 Migrate GPU pixel modules to ESConsumes
• Move cabling map files for PR#31721 cms-patatrack/cmssw#578 Move SiPixelFedCablingMapGPU to CondFormats and CalibTracker
• Remove Pilot Blade from GPU code cms-patatrack/cmssw#581 Remove partial handling of the Pilot Blade from GPU code
• Rename SiPixelFedCablingMapGPU to SiPixelROCsStatusAndMapping cms-patatrack/cmssw#582 Rename SiPixelFedCablingMapGPU to SiPixelROCsStatusAndMapping
• move to int32 for hit indices cms-patatrack/cmssw#583 Move hit indexes to 32 bits
• Update DQM clients for PixelVertexProducer pixel vertices cms-patatrack/cmssw#584 Update DQM clients for PixelVertexProducer pixel vertices
• Simplify cudacompat layer to use a 1-dimensional grid cms-patatrack/cmssw#586 Simplify cudacompat layer to use a 1-dimensional grid
• make clusterizer kernels independent of grid size cms-patatrack/cmssw#588 Make clusterizer kernels independent of the grid size
• Clean up the pixel local reconstruction code cms-patatrack/cmssw#593 Clean up the pixel local reconstruction code
• Clean up the pixel local reconstruction code cms-patatrack/cmssw#599 Clean up the pixel local reconstruction code
• Clean up the pixel local reconstruction code cms-patatrack/cmssw#601 Clean up the pixel local reconstruction code
• Clean up the pixel local reconstruction code cms-patatrack/cmssw#602 Clean up the pixel local reconstruction code

[cmsbuild]

The code-checks are being triggered in jenkins.

[fwyzard]

For all questions, please address @makortel @felicepantaleo @VinInn @rovere .
For all changes, please make PRs against cms-patatrack:patatrack_integration_9_N_pixel_local_reco .

[cmsbuild]

-code-checks

ERROR: Build errors found during clang-tidy run.

RecoLocalTracker/SiPixelRecHits/plugins/SiPixelRecHitCUDA.cc:7:10: error: 'CUDADataFormats/TrackingRecHit/interface/TrackingRecHit2DCUDA.h' file not found [clang-diagnostic-error]
#include "CUDADataFormats/TrackingRecHit/interface/TrackingRecHit2DCUDA.h"
         ^
Suppressed 1062 warnings (1061 in non-user code, 1 with check filters).
--
RecoLocalTracker/SiPixelRecHits/plugins/SiPixelRecHitFromSOA.cc:4:10: error: 'CUDADataFormats/Common/interface/HostProduct.h' file not found [clang-diagnostic-error]
#include "CUDADataFormats/Common/interface/HostProduct.h"
         ^
Suppressed 820 warnings (819 in non-user code, 1 with check filters).
--
RecoLocalTracker/SiPixelRecHits/plugins/SiPixelRecHitConverter.cc:87:10: error: 'CUDADataFormats/Common/interface/HostProduct.h' file not found [clang-diagnostic-error]
#include "CUDADataFormats/Common/interface/HostProduct.h"
         ^
Suppressed 1384 warnings (1383 in non-user code, 1 with check filters).
--
RecoLocalTracker/SiPixelRecHits/plugins/SiPixelRecHitSoAFromLegacy.cc:7:10: error: 'CUDADataFormats/TrackingRecHit/interface/TrackingRecHit2DHeterogeneous.h' file not found [clang-diagnostic-error]
#include "CUDADataFormats/TrackingRecHit/interface/TrackingRecHit2DHeterogeneous.h"
         ^
Suppressed 1102 warnings (1101 in non-user code, 1 with check filters).
--
gmake: *** [config/SCRAM/GMake/Makefile.coderules:128: code-checks] Error 2
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

[kpedro88]

+upgrade
(only one line changed in SLHCUpgradeSimulations/Geometry/test/phase2_digi_reco_pixelntuple_cfg.py)

[fwyzard]

@cms-sw/alca-l2 @cms-sw/db-l2 @cms-sw/geometry-l2 @cms-sw/simulation-l2 @cms-sw/dqm-l2
your signatures are missing for this PR.

If you were annoyed by the long review process and the large amount of comments and changes, sorry for that; we hope that this PR has finally settled down, so these could hopefully be your final signatures here.

[jfernan2]

+1

[silviodonato]

+operations

[silviodonato]

kind reminder @cms-sw/alca-l2 @cms-sw/db-l2 @cms-sw/geometry-l2 @cms-sw/simulation-l2

[cvuosalo]

+1

[ggovi]

+1

[civanch]

+1

[yuanchao]

+1

• Build tests and unit tests all passed
• Matrix tests all passed
• Build warnings on BuildFile.xml and HcalElectronicsId.h are not part of the modification

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[silviodonato]

+1