Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking developments for review and merging #338

Merged
merged 6 commits into from
May 15, 2019

Conversation

fwyzard
Copy link

@fwyzard fwyzard commented Apr 30, 2019

Split #324, part 7 of 8.

@fwyzard fwyzard added the Pixels Pixels-related developments label Apr 30, 2019
@@ -11,4 +11,5 @@
<use name="cuda"/>
<use name="cuda-api-wrappers"/>
<flags EDM_PLUGIN="1"/>
<flags CUDA_FLAGS="-g -DGPU_DEBUG"/>
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should not use these flags for a production plugin.

@@ -21,8 +21,10 @@ namespace trackerHitAssociationHeterogeneousProduct {

struct ClusterSLGPU {

using Clus2TP = std::array<uint32_t,6>;
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use a more meaningful or at least less abbreviated name, for example 'ClusterToTP' (assuming that is the meaning).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the meaning of 6 ?

@fwyzard
Copy link
Author

fwyzard commented May 2, 2019

Thanks to @rovere for cleaning up the commit history !

@fwyzard
Copy link
Author

fwyzard commented May 2, 2019

Filter spurious commits from upstream pull request cms-sw#26074 .

@fwyzard fwyzard force-pushed the ForReview_10_6_0_pre2 branch 3 times, most recently from a6dc1d2 to 4a028a1 Compare May 2, 2019 16:39
@fwyzard
Copy link
Author

fwyzard commented May 2, 2019

Clean up spurious commits that were already included in CMSSW_10_6_X_Patatrack

@fwyzard fwyzard force-pushed the ForReview_10_6_0_pre2 branch 3 times, most recently from 73dad12 to c0eea4f Compare May 2, 2019 20:54
@@ -17,6 +17,8 @@ class SiPixelGainCalibrationForHLTGPU {
~SiPixelGainCalibrationForHLTGPU();

const SiPixelGainForHLTonGPU *getGPUProductAsync(cuda::stream_t<>& cudaStream) const;
const SiPixelGainForHLTonGPU *getCPUProduct() const { return gainForHLTonHost_;}
const SiPixelGainCalibrationForHLT *getOriginalProduct() { return gains_;}
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these new methods used anywhere ?
Otherwise, do we (still) want them ?

@@ -26,7 +26,7 @@ SiPixelGainCalibrationForHLTGPU::SiPixelGainCalibrationForHLTGPU(const SiPixelGa
*/

cudaCheck(cudaMallocHost((void**) & gainForHLTonHost_, sizeof(SiPixelGainForHLTonGPU)));
//gainForHLTonHost_->v_pedestals = gainDataOnGPU_; // how to do this?
gainForHLTonHost_->v_pedestals = (SiPixelGainForHLTonGPU_DecodingStructure*)this->gains_->data().data(); // so it can be used on CPU as well...
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does this do ?

@fwyzard fwyzard force-pushed the ForReview_10_6_0_pre2 branch 6 times, most recently from 09de954 to 7686227 Compare May 3, 2019 17:02
@fwyzard
Copy link
Author

fwyzard commented May 3, 2019

current commit map

@fwyzard fwyzard force-pushed the ForReview_10_6_0_pre2 branch 3 times, most recently from bd977e4 to 25ac5c9 Compare May 10, 2019 11:46
fwyzard pushed a commit that referenced this pull request Dec 29, 2020
Add two alternative (faster) track clusterizers: one based on DBSCAN,
and one "by density"; use the latter by default.

Allow all pixel triplets, but protect the vertex from triplets.

Use a larger-then-needed nearest neightbours array to allow for possible
duplicate pixels, as a pixel can appear more than once in the same event.

Use a separate workspace for temporary data.
fwyzard pushed a commit that referenced this pull request Dec 29, 2020
Improve pixel doublets and CA:
  - add pixel cluster size and shape cuts in doublets;
  - add triplet cleaner;
  - improved cluster size studies
  - implement layer-dependent cuts in the CA.

Add counters in GPU code and possibility to test full doublet combinatorics.

Update python notebook and include z0 resolution.
fwyzard pushed a commit that referenced this pull request Dec 29, 2020
…amework (#338)

Use cleaned hits.
Use pixel layer and ladders geometry, and use pixel triplets in the gaps.

Optimise GPU memory usage:
  - reduce the number of memory allocations
  - fix the size of the cub workspace
  - allocate memory per event via the caching allocator
  - use constant memory for geometry and parameters
  - use shared memory where the content is the same for every thread

Optimise kernel launches, and add a protection for empty events and overflows.
fwyzard added a commit that referenced this pull request Dec 29, 2020
fwyzard pushed a commit that referenced this pull request Dec 29, 2020
Add two alternative (faster) track clusterizers: one based on DBSCAN,
and one "by density"; use the latter by default.

Allow all pixel triplets, but protect the vertex from triplets.

Use a larger-then-needed nearest neightbours array to allow for possible
duplicate pixels, as a pixel can appear more than once in the same event.

Use a separate workspace for temporary data.
fwyzard pushed a commit that referenced this pull request Dec 29, 2020
Improve pixel doublets and CA:
  - add pixel cluster size and shape cuts in doublets;
  - add triplet cleaner;
  - improved cluster size studies
  - implement layer-dependent cuts in the CA.

Add counters in GPU code and possibility to test full doublet combinatorics.

Update python notebook and include z0 resolution.
fwyzard pushed a commit that referenced this pull request Dec 29, 2020
…amework (#338)

Use cleaned hits.
Use pixel layer and ladders geometry, and use pixel triplets in the gaps.

Optimise GPU memory usage:
  - reduce the number of memory allocations
  - fix the size of the cub workspace
  - allocate memory per event via the caching allocator
  - use constant memory for geometry and parameters
  - use shared memory where the content is the same for every thread

Optimise kernel launches, and add a protection for empty events and overflows.
fwyzard added a commit that referenced this pull request Dec 29, 2020
fwyzard pushed a commit that referenced this pull request Dec 29, 2020
Merge the Riemann and broken line fits into single configurable pixel
n-tuplet fitter, and extend it to work with up to 5 hits.

Mmake the broken line fit the default algorithm.

Try both triplets and quadruplets in the pixel "hole".

Limit pT used to compute the multple scattering.

Use the inline Cholesky decomposition.

Generic clean up and improvements.
fwyzard pushed a commit that referenced this pull request Dec 29, 2020
Add two alternative (faster) track clusterizers: one based on DBSCAN,
and one "by density"; use the latter by default.

Allow all pixel triplets, but protect the vertex from triplets.

Use a larger-then-needed nearest neightbours array to allow for possible
duplicate pixels, as a pixel can appear more than once in the same event.

Use a separate workspace for temporary data.
fwyzard added a commit that referenced this pull request Dec 29, 2020
fwyzard pushed a commit that referenced this pull request Jan 13, 2021
Add two alternative (faster) track clusterizers: one based on DBSCAN,
and one "by density"; use the latter by default.

Allow all pixel triplets, but protect the vertex from triplets.

Use a larger-then-needed nearest neightbours array to allow for possible
duplicate pixels, as a pixel can appear more than once in the same event.

Use a separate workspace for temporary data.
fwyzard pushed a commit that referenced this pull request Jan 13, 2021
fwyzard pushed a commit that referenced this pull request Jan 15, 2021
Add two alternative (faster) track clusterizers: one based on DBSCAN,
and one "by density"; use the latter by default.

Allow all pixel triplets, but protect the vertex from triplets.

Use a larger-then-needed nearest neightbours array to allow for possible
duplicate pixels, as a pixel can appear more than once in the same event.

Use a separate workspace for temporary data.
fwyzard added a commit that referenced this pull request Jan 15, 2021
fwyzard pushed a commit that referenced this pull request Jan 15, 2021
Merge the Riemann and broken line fits into single configurable pixel
n-tuplet fitter, and extend it to work with up to 5 hits.

Mmake the broken line fit the default algorithm.

Try both triplets and quadruplets in the pixel "hole".

Limit pT used to compute the multple scattering.

Use the inline Cholesky decomposition.

Generic clean up and improvements.
fwyzard pushed a commit that referenced this pull request Jan 15, 2021
Improve pixel doublets and CA:
  - add pixel cluster size and shape cuts in doublets;
  - add triplet cleaner;
  - improved cluster size studies
  - implement layer-dependent cuts in the CA.

Add counters in GPU code and possibility to test full doublet combinatorics.

Update python notebook and include z0 resolution.
fwyzard pushed a commit that referenced this pull request Jan 15, 2021
…amework (#338)

Use cleaned hits.
Use pixel layer and ladders geometry, and use pixel triplets in the gaps.

Optimise GPU memory usage:
  - reduce the number of memory allocations
  - fix the size of the cub workspace
  - allocate memory per event via the caching allocator
  - use constant memory for geometry and parameters
  - use shared memory where the content is the same for every thread

Optimise kernel launches, and add a protection for empty events and overflows.
fwyzard added a commit that referenced this pull request Jan 15, 2021
fwyzard pushed a commit that referenced this pull request Mar 23, 2021
Merge the Riemann and broken line fits into single configurable pixel
n-tuplet fitter, and extend it to work with up to 5 hits.

Mmake the broken line fit the default algorithm.

Try both triplets and quadruplets in the pixel "hole".

Limit pT used to compute the multple scattering.

Use the inline Cholesky decomposition.

Generic clean up and improvements.
fwyzard pushed a commit that referenced this pull request Mar 23, 2021
Improve pixel doublets and CA:
  - add pixel cluster size and shape cuts in doublets;
  - add triplet cleaner;
  - improved cluster size studies
  - implement layer-dependent cuts in the CA.

Add counters in GPU code and possibility to test full doublet combinatorics.

Update python notebook and include z0 resolution.
fwyzard pushed a commit that referenced this pull request Mar 23, 2021
…amework (#338)

Use cleaned hits.
Use pixel layer and ladders geometry, and use pixel triplets in the gaps.

Optimise GPU memory usage:
  - reduce the number of memory allocations
  - fix the size of the cub workspace
  - allocate memory per event via the caching allocator
  - use constant memory for geometry and parameters
  - use shared memory where the content is the same for every thread

Optimise kernel launches, and add a protection for empty events and overflows.
fwyzard added a commit that referenced this pull request Mar 23, 2021
fwyzard pushed a commit that referenced this pull request Apr 1, 2021
Merge the Riemann and broken line fits into single configurable pixel
n-tuplet fitter, and extend it to work with up to 5 hits.

Mmake the broken line fit the default algorithm.

Try both triplets and quadruplets in the pixel "hole".

Limit pT used to compute the multple scattering.

Use the inline Cholesky decomposition.

Generic clean up and improvements.
fwyzard pushed a commit that referenced this pull request Apr 1, 2021
Improve pixel doublets and CA:
  - add pixel cluster size and shape cuts in doublets;
  - add triplet cleaner;
  - improved cluster size studies
  - implement layer-dependent cuts in the CA.

Add counters in GPU code and possibility to test full doublet combinatorics.

Update python notebook and include z0 resolution.
fwyzard pushed a commit that referenced this pull request Apr 1, 2021
…amework (#338)

Use cleaned hits.
Use pixel layer and ladders geometry, and use pixel triplets in the gaps.

Optimise GPU memory usage:
  - reduce the number of memory allocations
  - fix the size of the cub workspace
  - allocate memory per event via the caching allocator
  - use constant memory for geometry and parameters
  - use shared memory where the content is the same for every thread

Optimise kernel launches, and add a protection for empty events and overflows.
fwyzard added a commit that referenced this pull request Apr 1, 2021
silviodonato pushed a commit to cms-sw/cmssw that referenced this pull request Apr 12, 2021
* Add workflows for Riemann fit and GPU (cms-patatrack#20)

* add `riemannFit` and `gpu` modifiers and workflows for Riemann fit and GPU modules
* switch GPU modules with `gpu` modifier

* Add a DQM sequence for pixel-only tracking (cms-patatrack#23)

* add a DQM sequence for pixel-only tracking
* add pixelTrackingOnlyDQM sequences to the pixelTrackingOnly workflows
* add pixelTrackingOnlyDQM sequences to the riemannFit and gpu workflows

* Synchronise with CMSSW_10_1_0_pre2

* Synchronise with CMSSW_10_1_0_pre3

* Synchronise with CMSSW_10_1_0

* Synchronise with CMSSW_10_2_0_pre1

* Synchronise with CMSSW_10_2_0_pre2

* Synchronise with CMSSW_10_2_0_pre3

* Synchronise with CMSSW_10_2_0_pre4

* Synchronise with CMSSW_10_2_0_pre5

* Port the Riemann fit to CUDA (cms-patatrack#60)

  - the CPU Riemann fit works using all combinations between the 2 booleans: `useErrors` and `useMultipleScattering`;
  - the standalone version of the GPU Riemann fit has been updated in order to explore all possibilities among the 2 booleans above: all of them work and produce identical results up to 1e-5 precision (the default one, 1e-6 fails when enabling multiScattering, most likely due to matrix inversions);
  - the GPU version of the Riemann fit within CMSSW works, with 1 fit assigned to each thread, with 32 threads/warps, all dynamically computed.

Things that needs a "hack":

  - limit the "dynamic" size of Eigen matrices to at most, 4x4, which is just fine for quadruplets. Using anything wider will cause errors which I *believe* is related to the stack size of threads on the GPU;
  - cast matrices to be inverted to 4x4 (was done before the previous point: will revert it back and see if that's still needed or not, but I believe it is); this was done in order to "specialize" the `invert()` call to something that is "natively" supported by Eigen on GPU (that brought in also few `__host__` `__device__` here and there in Eigen);
  - fix the alignment of the `struct` holding the results of the fit, since its size was different on GPU and CPU, causing an annoying off-by-one effect.

* Synchronise with CMSSW_10_2_0_pre6

* update relval input strings (102X)

* Synchronise with CMSSW_10_2_0

* adding NANO step to 2018 MC workflows

* Merge pull request #24024 from prebello/from-CMSSW_10_2_X_2018-07-24-2300

update relval input strings (102X)

* Synchronise with CMSSW_10_2_2_patch1

* activate the new SiStripCalSiStripCalBiasScan in the matrix test

* Add pixel tracking workflows for data (cms-patatrack#144)

* add HEM-15-16 relval workflows

* introduce RelVal for hybrid emulation (with ZS10 repacking),HLT-like zero-suppression (with normal ZS repacking),RECO of 2015 PbPb VirginRaw data

* add 3 production workflows for premix2018

* Merge pull request #24550 from zhenhu/add-HEM-15-16-relvals_102X

[10.2.X] add HEM-15-16 relval workflows

* Merge pull request #24618 from zhenhu/addPremixProdWfs_102X

backport [10.2.X] add 3 production workflows for premix2018

* Synchronise with CMSSW_10_2_5

* Synchronise with CMSSW_10_2_6

* Synchronise with CMSSW_10_4_0_pre2

* Synchronise with CMSSW_10_4_0_pre3

* Synchronise with CMSSW_10_4_0_pre4

* Synchronise with CMSSW_10_4_0

* Change GPU workflow numbering: .7->.51, .8->.52, .9->.53 (cms-patatrack#259)

* Synchronise with CMSSW_10_5_0_pre1

* Synchronise with CMSSW_10_5_0_pre2

* Synchronise with CMSSW_10_6_0_pre2

* Rework the Riemann fit and broken line fit (cms-patatrack#338)

Merge the Riemann and broken line fits into single configurable pixel
n-tuplet fitter, and extend it to work with up to 5 hits.

Mmake the broken line fit the default algorithm.

Try both triplets and quadruplets in the pixel "hole".

Limit pT used to compute the multple scattering.

Use the inline Cholesky decomposition.

Generic clean up and improvements.

* Synchronise with CMSSW_10_6_0

* Synchronise with CMSSW_11_0_0_pre7

* Fix broken merge in Configuration/PyReleaseValidation

* Synchronise with CMSSW_11_0_0_pre11

* Update and extend the Patatrack workflows

Update the definition of the Patatrack workflows to avoid conflicts in CMSSW_11_0_0_pre12:
  - renumber to avoid conflicts with #27392
  - reimplement the definition following #28281
  - add Z->mumu 2018 realistic workflows
  - add TTbar and Z->mumu 2021 realistic workflows
  - move the data workflows to 2018D, HLTPhysics and JetHT

* Synchronise with CMSSW_11_0_0_pre13

* Backport: add ECAL-only workflows for 2018 data (#30105)

* Add tracking-only and pixel tracking-only workflows for Run 3 (#30338)

* Backport: add ECAL-only and HCAL-only workflows for MC and data (#30350)

Backport #30105: add ECAL-only workflows for data.
Backport #30136: add HCAL-only workflows for MC and data.

* Synchronise with CMSSW_11_2_0_pre2

* Update the RelVal workflows and the CPU customisation (cms-patatrack#549)

Update the RelVal workflows and the CPU customisation:
  - change the .501 workflow to run the full Patatrack pixel track reconstruction on CPU
  - add a customisation to run the Patatrack reconstruction with triplets, on CPU and GPU
  - add the .505 and .506 workflows to reconstruct triplets, on CPU and GPU

Co-authored-by: Andrea Bocci <andrea.bocci@cern.ch>

* Synchronise with CMSSW_11_2_0_pre8

* Synchronise with CMSSW_11_2_0_pre9

* Synchronise with CMSSW_11_2_0_pre10

* Backport: add RecoFakeHLT steps to 2018 Patatrack WFs to enable GPU modifiers (#32333)

The 2018 GPU matrix WFs (10824.502, 10824.512, 10824.522, 10842.502) produced the same cmsDriver configurations as their CPU counterparts. This PR fixes this by setting the gpu modifier also for the RecoFakeHLT step.

* Synchronise with CMSSW_11_3_0_pre1

* Synchronise with CMSSW_11_3_X_2020-12-24-2300

* Document the new Pixel workflows

Co-authored-by: Matti Kortelainen <matti.kortelainen@cern.ch>
Co-authored-by: Marco Rovere <rovere@users.noreply.github.com>
Co-authored-by: Patricia <patricia.rebello.teles@cern.ch>
Co-authored-by: Francesco <Francesco.Fabozzi@cern.ch>
Co-authored-by: cmsbuild <cmsbuild@cern.ch>
Co-authored-by: mmusich <marco.musich@cern.ch>
Co-authored-by: zhen <zhen.hu@cern.ch>
Co-authored-by: Vincenzo Innocente <vincenzo.innocente@cern.ch>
Co-authored-by: Mariarosaria D'Alfonso <dalfonso@cern.ch>
Co-authored-by: AdrianoDee <AdrianoDee@users.noreply.github.com>
Co-authored-by: Thomas Reis <thomas.reis@cern.ch>
marco-link added a commit to marco-link/cmssw that referenced this pull request Apr 21, 2021
* Code check

* Add PPS geometry and simulation tags to GTs

* Changing the proton propagator to the Optical Function method

* Revert back to standard naming for Phase1 classes

* Fixed include guards

* Adding the new filter modules

* Code format

* Remove unused debug for-loop.

* Adding the multiple daughters of same kind possibility

* Use esConsumes for geometry and ETTmap records.

* Codecheck update

* Fix the deug statements

* use old hadronic physics

* Correcting the clang build err
	- corrected splelling in header guard

* fixed compilation

* add 14 TeV VBF H>inv pythia cfg

* added update on low-energy GFlash

* fix issues with clang tidy and static analyzer

* First skeleton of a Trackster from Simcluster producer

* Fix bug

* Add simtracksters into the event content

* Make SimTracksters part of TICL Tasks

* Create a SimTrackster per SimCluster

* Deleted methods should be public

* keep multiclusters from simtracksters

* cleanup

* add multiclusters from simtrackstrs

* add simtracksters to validation

* Make multiplicity take the simulated energy fraction

* add simTracksters iteration to validation

* replacing tabs with spaces and speeding up --help

* make enabling simclusters validation plots

* cleanup

* Add SimTracksters to FEVT and PreValidation Sequence

* Fix for premix wf

* get ES data through token

* update and reorder BuildFile

* adding missing dependencies in BuildFile

* Patatrack integration - Pixel workflows (12/N) (cms-sw#31854)

* Add workflows for Riemann fit and GPU (cms-patatrack#20)

* add `riemannFit` and `gpu` modifiers and workflows for Riemann fit and GPU modules
* switch GPU modules with `gpu` modifier

* Add a DQM sequence for pixel-only tracking (cms-patatrack#23)

* add a DQM sequence for pixel-only tracking
* add pixelTrackingOnlyDQM sequences to the pixelTrackingOnly workflows
* add pixelTrackingOnlyDQM sequences to the riemannFit and gpu workflows

* Synchronise with CMSSW_10_1_0_pre2

* Synchronise with CMSSW_10_1_0_pre3

* Synchronise with CMSSW_10_1_0

* Synchronise with CMSSW_10_2_0_pre1

* Synchronise with CMSSW_10_2_0_pre2

* Synchronise with CMSSW_10_2_0_pre3

* Synchronise with CMSSW_10_2_0_pre4

* Synchronise with CMSSW_10_2_0_pre5

* Port the Riemann fit to CUDA (cms-patatrack#60)

  - the CPU Riemann fit works using all combinations between the 2 booleans: `useErrors` and `useMultipleScattering`;
  - the standalone version of the GPU Riemann fit has been updated in order to explore all possibilities among the 2 booleans above: all of them work and produce identical results up to 1e-5 precision (the default one, 1e-6 fails when enabling multiScattering, most likely due to matrix inversions);
  - the GPU version of the Riemann fit within CMSSW works, with 1 fit assigned to each thread, with 32 threads/warps, all dynamically computed.

Things that needs a "hack":

  - limit the "dynamic" size of Eigen matrices to at most, 4x4, which is just fine for quadruplets. Using anything wider will cause errors which I *believe* is related to the stack size of threads on the GPU;
  - cast matrices to be inverted to 4x4 (was done before the previous point: will revert it back and see if that's still needed or not, but I believe it is); this was done in order to "specialize" the `invert()` call to something that is "natively" supported by Eigen on GPU (that brought in also few `__host__` `__device__` here and there in Eigen);
  - fix the alignment of the `struct` holding the results of the fit, since its size was different on GPU and CPU, causing an annoying off-by-one effect.

* Synchronise with CMSSW_10_2_0_pre6

* update relval input strings (102X)

* Synchronise with CMSSW_10_2_0

* adding NANO step to 2018 MC workflows

* Merge pull request cms-sw#24024 from prebello/from-CMSSW_10_2_X_2018-07-24-2300

update relval input strings (102X)

* Synchronise with CMSSW_10_2_2_patch1

* activate the new SiStripCalSiStripCalBiasScan in the matrix test

* Add pixel tracking workflows for data (cms-patatrack#144)

* add HEM-15-16 relval workflows

* introduce RelVal for hybrid emulation (with ZS10 repacking),HLT-like zero-suppression (with normal ZS repacking),RECO of 2015 PbPb VirginRaw data

* add 3 production workflows for premix2018

* Merge pull request cms-sw#24550 from zhenhu/add-HEM-15-16-relvals_102X

[10.2.X] add HEM-15-16 relval workflows

* Merge pull request cms-sw#24618 from zhenhu/addPremixProdWfs_102X

backport [10.2.X] add 3 production workflows for premix2018

* Synchronise with CMSSW_10_2_5

* Synchronise with CMSSW_10_2_6

* Synchronise with CMSSW_10_4_0_pre2

* Synchronise with CMSSW_10_4_0_pre3

* Synchronise with CMSSW_10_4_0_pre4

* Synchronise with CMSSW_10_4_0

* Change GPU workflow numbering: .7->.51, .8->.52, .9->.53 (cms-patatrack#259)

* Synchronise with CMSSW_10_5_0_pre1

* Synchronise with CMSSW_10_5_0_pre2

* Synchronise with CMSSW_10_6_0_pre2

* Rework the Riemann fit and broken line fit (cms-patatrack#338)

Merge the Riemann and broken line fits into single configurable pixel
n-tuplet fitter, and extend it to work with up to 5 hits.

Mmake the broken line fit the default algorithm.

Try both triplets and quadruplets in the pixel "hole".

Limit pT used to compute the multple scattering.

Use the inline Cholesky decomposition.

Generic clean up and improvements.

* Synchronise with CMSSW_10_6_0

* Synchronise with CMSSW_11_0_0_pre7

* Fix broken merge in Configuration/PyReleaseValidation

* Synchronise with CMSSW_11_0_0_pre11

* Update and extend the Patatrack workflows

Update the definition of the Patatrack workflows to avoid conflicts in CMSSW_11_0_0_pre12:
  - renumber to avoid conflicts with cms-sw#27392
  - reimplement the definition following cms-sw#28281
  - add Z->mumu 2018 realistic workflows
  - add TTbar and Z->mumu 2021 realistic workflows
  - move the data workflows to 2018D, HLTPhysics and JetHT

* Synchronise with CMSSW_11_0_0_pre13

* Backport: add ECAL-only workflows for 2018 data (cms-sw#30105)

* Add tracking-only and pixel tracking-only workflows for Run 3 (cms-sw#30338)

* Backport: add ECAL-only and HCAL-only workflows for MC and data (cms-sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.

* Synchronise with CMSSW_11_2_0_pre2

* Update the RelVal workflows and the CPU customisation (cms-patatrack#549)

Update the RelVal workflows and the CPU customisation:
  - change the .501 workflow to run the full Patatrack pixel track reconstruction on CPU
  - add a customisation to run the Patatrack reconstruction with triplets, on CPU and GPU
  - add the .505 and .506 workflows to reconstruct triplets, on CPU and GPU

Co-authored-by: Andrea Bocci <andrea.bocci@cern.ch>

* Synchronise with CMSSW_11_2_0_pre8

* Synchronise with CMSSW_11_2_0_pre9

* Synchronise with CMSSW_11_2_0_pre10

* Backport: add RecoFakeHLT steps to 2018 Patatrack WFs to enable GPU modifiers (cms-sw#32333)

The 2018 GPU matrix WFs (10824.502, 10824.512, 10824.522, 10842.502) produced the same cmsDriver configurations as their CPU counterparts. This PR fixes this by setting the gpu modifier also for the RecoFakeHLT step.

* Synchronise with CMSSW_11_3_0_pre1

* Synchronise with CMSSW_11_3_X_2020-12-24-2300

* Document the new Pixel workflows

Co-authored-by: Matti Kortelainen <matti.kortelainen@cern.ch>
Co-authored-by: Marco Rovere <rovere@users.noreply.github.com>
Co-authored-by: Patricia <patricia.rebello.teles@cern.ch>
Co-authored-by: Francesco <Francesco.Fabozzi@cern.ch>
Co-authored-by: cmsbuild <cmsbuild@cern.ch>
Co-authored-by: mmusich <marco.musich@cern.ch>
Co-authored-by: zhen <zhen.hu@cern.ch>
Co-authored-by: Vincenzo Innocente <vincenzo.innocente@cern.ch>
Co-authored-by: Mariarosaria D'Alfonso <dalfonso@cern.ch>
Co-authored-by: AdrianoDee <AdrianoDee@users.noreply.github.com>
Co-authored-by: Thomas Reis <thomas.reis@cern.ch>

* Remove CLCT BX and LCT BX comparisons

* add valueMap producer for the track distances

* add monitoring of closest track

* use CTPPSRun2Geometry customization to fix HLT test with Run2 data

* use the iEvent.getHandle syntax and make TrackDistanceValueMapProducer a global EDProducer

* Use 112X geometry in online data global tags

* Use 112X geometry in offline data global tags

* fix in CTPPSRun2Geometry cust funt

* fix in CTPPSRun2Geometry cust funt

* fix HLT addOnTest to use CTPPSRun2Geometry custom funct

* Disable the deug possibility

* add 14 tev gluino model

* Remove trackingMaterialGroups_ForPhaseII.xml from T25/T26 dict

* add comment to deprecate Multiclusters

* add name and date

* Modernize code and simplify

* add the possibility to save only the first nth closest track distances

* fix container size

* Add a new workflow with V15 version of HGCal Geometry

* Modify Run3 workflows with changed version of RP devices

* add spaces

* Change nodigis to hasdigis

* Hit counter for shower trigger

* Use constants from CSCConstants

* Use constants from CSCConstants

* Added ZpEE and ZpMM 14TeV fragments

* Use auto

* Migrate EcalTPCondAnalyzer to use esConsumes and replace couts with LogVerbatim.

* Move CSCConstants.h to DataFormats/CSCDigi

* Better document the wiregroups

* Delete particleFlowBadHcalPseudoCluster_cff.py

Was added by accident instead of RecoParticleFlow/PFClusterProducer/python/particleFlowBadHcalPseudoCluster_cff.py

* migrate DQM/TrackingMonitor to esConsumes

* Move plugin out of SimDataFormats

* Fix the memory error

* Code format

* First skeletton of the harvester

* Added skeletton for the worker

* Starting to populate a few monitors

* Making use of detid's naming export features

* Updated to the new PPS DataFormat accessors

* Stripping the histogram containers from worker

* Using rechits instead of digis

* Filling the condition on pixel tracks

* Full implementation of pixels tracks selection stage

* Corrected RP station and pixel tracks cuts retrieval

* Starting to integrate geometry

* Full usage of db-loaded geometry for channels booking, starting to fill the harvester

* Dropped CLHEP dependency

* Also storing uncalibrated ToT distribution, first attempt to fit the t-vs-tot profile in harvester

* Applied code-format

* Inserted PPSTimingCalib selection/paths/output sequences in their right place

* Delete ME file after copy

* Small cleanup, filtered stream renaming

* Reverted back filtered stream name

* Matched new folders hierarchy

* Updated paths

* Added Geometry/Records to TimingCalibration build

* Added an uncalibrated/calibrated collection of diamond information

* PPS track filter stripped from PCL worker

* Added a test sequence wrapping up all PCL operations

* Cleanup

* Per-pot track filter for PPS

* More C++-friendly tracks counting

* Replaced the track filter by the HLT-defined one

* Cleanup

* Cleanup of timing calibration worker sequence

* Keeping the diamond digis instead of rechits to perform uncalibrated rechits "reconstruction"

* Cleanup of harvester includes

* Using a smart pointer to hold x-profile used in calibration fit

* Ensure PoolDBService is available before processing

* Delegating DB service availability check at plugin constructor level

* Avoiding a couple of try/catch blocks through CTPPSDiamondDetId::check

* Filters rules retrieved by reference instead of copies

* Thread-safe geometry retrieval from ES

* Code-format

* Dropping the DB service private attribute to create it at constructor and for each end job

* Transition type set for geometry consumes declaration

* Thread-safe ES geometry condition retrieval for PCL harvester

* Update CalibPPS/TimingCalibration/python/ALCARECOPPSCalTrackBasedSel_Output_cff.py

Co-authored-by: Silvio Donato <silvio.donato@cern.ch>

* Update CalibPPS/TimingCalibration/python/ALCARECOPPSCalTrackBasedSel_cff.py

Co-authored-by: Silvio Donato <silvio.donato@cern.ch>

* Cloning instead of deep-copying, Task/Sequence redefinition of the filtering/worker modules

* Replaced back ','s with '*'s for seqALCARECOPPSCalTrackBasedSel Sequence

* Using a reco task instead of a sequence for the taskALCARECOPPSTimingCalib task definition

* PCL WF test is now 1041

* Apply changes from : scram build code-checks / code-format

* use references

* Correct validation code for MB of HCAL

* Code check

* Utilize a phase2 scenario for testing dd4hep scenarios

* Clarify number of strips and half-strips

* Tighten assert statements

* put distrip and cfeb calculation in functions

* Fix typo

* Check in more detail the content of teh geometry

* Code check

* Clean BuildFiles

* rename and small fixes

* CLANG test

* cleanup of Track analyzer moving double loop to dqm harvester

* remove commented lines

* remove unwanted includes and cout usage

* remove duplicate includes and cout

* resolve code check issues

* Move the macros to a separate directory and adopt for dd4hep

* Tryto fix

* change logic in folded occupancy client to allow running for cosmics;update cosmics config

* Code check

* update HI sequence

* apply coding style

* reorganization in configs

* Add step3 and 4 for TICL only

* Move check for pixel active area into RectangularMTDTopology, make it usable for BTL as well

* Fix post merge compilation errors

* Code format

* Fix indentation in EDAlias python dump

* Add a new scenario D84 same as D70 but with updated tracker T24

* updated physics to 10.7

* removed obsolete

* removed remaining obsolete

* complete update

* complete update

* clean-up custom physics

* code-checks

* code-checks

* update input file in fwliteWithPythonConfig_cfg.py

* Remove dead essignments from L1Trigger/CSCTrackFinder

* Suggestions by Andrea Perrotta

* Suggestions by Andrea Perrotta

* Modified version of EB numbering scheme code which will work with modified EB algorithm

* deal with fastsim issue raised in cms-sw#33430

* Code check

* Update python tests - py3 only

* Code format

* use python3 for all tests

* Minor changes in the configs removing uneeded loads in view of the new GT

* uncommented the verbosity line and changed the names

* fixed the names

* correct for gen fragment naming conventions

* remove old cfgs

Co-authored-by: Sunanda <sunanda.banerjee@cern.ch>
Co-authored-by: Helena <helena.malbouisson@gmail.com>
Co-authored-by: cmsbuild <cmsbuild@cern.ch>
Co-authored-by: Luiz Mundim <luiz.mundim@cern.ch>
Co-authored-by: Davide Valsecchi <davide.valsecchi@cern.ch>
Co-authored-by: Aravind <aravindsugunan@gmail.com>
Co-authored-by: Thomas Reis <thomas.reis@cern.ch>
Co-authored-by: Vladimir <Vladimir.Ivantchenko@cern.ch>
Co-authored-by: Vincenzo Innocente <vincenzo.innocente@cern.ch>
Co-authored-by: Silvio <silvio.donato@cern.ch>
Co-authored-by: Theodor Christian Herwig <2671-therwig@users.noreply.gitlab.cern.ch>
Co-authored-by: mmusich <marco.musich@cern.ch>
Co-authored-by: Felice <felice.pantaleo@cern.ch>
Co-authored-by: Andrea Bocci <andrea.bocci@cern.ch>
Co-authored-by: Matti Kortelainen <matti.kortelainen@cern.ch>
Co-authored-by: Marco Rovere <rovere@users.noreply.github.com>
Co-authored-by: Patricia <patricia.rebello.teles@cern.ch>
Co-authored-by: Francesco <Francesco.Fabozzi@cern.ch>
Co-authored-by: zhen <zhen.hu@cern.ch>
Co-authored-by: Mariarosaria D'Alfonso <dalfonso@cern.ch>
Co-authored-by: AdrianoDee <AdrianoDee@users.noreply.github.com>
Co-authored-by: Sven Dildick <sven.dildick@cern.ch>
Co-authored-by: Chris West <christopher.alan.west@gmail.com>
Co-authored-by: Adinda de Wit <adinda.maite.de.wit@cern.ch>
Co-authored-by: Andrea <perrotta@cern.ch>
Co-authored-by: namppl <kyungwook.nam@cern.ch>
Co-authored-by: laurenhay <lmhay@buffalo.edu>
Co-authored-by: Leonardo Cristella <leonardo.cristella@cern.ch>
Co-authored-by: Laurent Forthomme <laurent.forthomme@cern.ch>
Co-authored-by: Laurent Forthomme <lforthomme@protonmail.com>
Co-authored-by: Carlo Battilana <carlo.battilana@gmail.com>
Co-authored-by: Jonas Rembser <jonas.rembser@cern.ch>
Co-authored-by: Suvankar Roy Chowdhury <suvankar.roy.chowdhury@cern.ch>
Co-authored-by: Erica Brondolin <erica.brondolin@cern.ch>
Co-authored-by: Fabio Cossutti <fabio.cossutti@ts.infn.it>
Co-authored-by: Mircho Rodozov <mrodozov@cern.ch>
Co-authored-by: Mircho Nikolaev Rodozov <mrodozov@cmsdev25.cern.ch>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Pixels Pixels-related developments
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants