Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ECAL-only workflows for 2018 data #30105

Merged
merged 1 commit into from Jun 5, 2020

Conversation

fwyzard
Copy link
Contributor

@fwyzard fwyzard commented Jun 4, 2020

PR description:

Implement ECAL-only local reconstruction workflows runnig over 2018D data, using the same sequences as the similar MC workflows (minus the MC-based validation):

  • 136.885511: 2018D HLTPhysics dataset, ECAL-only running on CPU
  • 136.885512: 2018D HLTPhysics dataset, ECAL-only running on GPU (currently a placeholder)
  • 136.888511: 2018D JetHT dataset, ECAL-only running on CPU
  • 136.888512: 2018D JetHT dataset, ECAL-only running on GPU (currently a placeholder)

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 4, 2020

The code-checks are being triggered in jenkins.

@fwyzard
Copy link
Contributor Author

fwyzard commented Jun 4, 2020

@cmsbuild, please test workflow 136.885511, 136.885512, 136.888511, 136.888512

@fwyzard
Copy link
Contributor Author

fwyzard commented Jun 4, 2020

@rchatter @amassiro FYI

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 4, 2020

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30105/15855

  • This PR adds an extra 56KB to repository

  • There are other open Pull requests which might conflict with changes you have proposed:

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 4, 2020

The tests are being triggered in jenkins.
Test Parameters:

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 4, 2020

A new Pull Request was created by @fwyzard (Andrea Bocci) for master.

It involves the following packages:

Configuration/PyReleaseValidation

@chayanit, @cmsbuild, @wajidalikhan, @pgunnell, @kpedro88 can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @fabiocos, @slomeo this is something you requested to watch as well.
@silviodonato, @dpiparo you are the release manager for this.

cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 4, 2020

+1
Tested at: 9732110
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6f4f8a/6799/summary.html
CMSSW: CMSSW_11_2_X_2020-06-03-2300
SCRAM_ARCH: slc7_amd64_gcc820

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 4, 2020

Comparison job queued.

fwyzard added a commit to cms-patatrack/cmssw that referenced this pull request Jun 4, 2020
fwyzard added a commit to cms-patatrack/cmssw that referenced this pull request Jun 4, 2020
@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 4, 2020

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6f4f8a/6799/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6f4f8a/136.885511_RunHLTPhy2018D+RunHLTPhy2018D+HLTDR2_2018+RECODR2_2018reHLT_ECALOnlyCPU+HARVEST2018_ECALOnly
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6f4f8a/136.885512_RunHLTPhy2018D+RunHLTPhy2018D+HLTDR2_2018+RECODR2_2018reHLT_ECALOnlyGPU+HARVEST2018_ECALOnly
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6f4f8a/136.888511_RunJetHT2018D+RunJetHT2018D+HLTDR2_2018+RECODR2_2018reHLT_ECALOnlyCPU+HARVEST2018_ECALOnly
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6f4f8a/136.888512_RunJetHT2018D+RunJetHT2018D+HLTDR2_2018+RECODR2_2018reHLT_ECALOnlyGPU+HARVEST2018_ECALOnly

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 2 differences found in the comparisons
  • DQMHistoTests: Total files compared: 36
  • DQMHistoTests: Total histograms compared: 2783946
  • DQMHistoTests: Total failures: 51
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2783845
  • DQMHistoTests: Total skipped: 50
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 35 files compared)
  • Checked 152 log files, 16 edm output root files, 36 DQM output files

@rchatter
Copy link

rchatter commented Jun 4, 2020

@rchatter @amassiro FYI

Thanks Andrea!

@kpedro88
Copy link
Contributor

kpedro88 commented Jun 4, 2020

+upgrade

fwyzard pushed a commit to cms-patatrack/cmssw that referenced this pull request Nov 6, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard added a commit to cms-patatrack/cmssw that referenced this pull request Nov 6, 2020
fwyzard pushed a commit to cms-patatrack/cmssw that referenced this pull request Nov 6, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard pushed a commit to cms-patatrack/cmssw that referenced this pull request Nov 9, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard pushed a commit to cms-patatrack/cmssw that referenced this pull request Nov 12, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard pushed a commit to cms-patatrack/cmssw that referenced this pull request Nov 16, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard added a commit to cms-patatrack/cmssw that referenced this pull request Nov 16, 2020
fwyzard pushed a commit to cms-patatrack/cmssw that referenced this pull request Nov 16, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard pushed a commit to cms-patatrack/cmssw that referenced this pull request Nov 16, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard pushed a commit to cms-patatrack/cmssw that referenced this pull request Nov 16, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard added a commit to cms-patatrack/cmssw that referenced this pull request Nov 26, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard added a commit to cms-patatrack/cmssw that referenced this pull request Nov 27, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard added a commit to cms-patatrack/cmssw that referenced this pull request Nov 27, 2020
fwyzard added a commit to cms-patatrack/cmssw that referenced this pull request Nov 27, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard added a commit to cms-patatrack/cmssw that referenced this pull request Nov 27, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard pushed a commit to cms-patatrack/cmssw that referenced this pull request Dec 25, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard pushed a commit to cms-patatrack/cmssw that referenced this pull request Dec 25, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard added a commit to cms-patatrack/cmssw that referenced this pull request Dec 26, 2020
fwyzard added a commit to cms-patatrack/cmssw that referenced this pull request Dec 26, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard added a commit to cms-patatrack/cmssw that referenced this pull request Dec 26, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard pushed a commit to cms-patatrack/cmssw that referenced this pull request Dec 29, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard pushed a commit to cms-patatrack/cmssw that referenced this pull request Dec 29, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard pushed a commit to cms-patatrack/cmssw that referenced this pull request Dec 29, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard added a commit to cms-patatrack/cmssw that referenced this pull request Dec 29, 2020
fwyzard pushed a commit to cms-patatrack/cmssw that referenced this pull request Dec 29, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard pushed a commit to cms-patatrack/cmssw that referenced this pull request Dec 29, 2020
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard pushed a commit to cms-patatrack/cmssw that referenced this pull request Jan 13, 2021
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
fwyzard pushed a commit to cms-patatrack/cmssw that referenced this pull request Jan 15, 2021
…sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.
silviodonato pushed a commit that referenced this pull request Apr 12, 2021
* Add workflows for Riemann fit and GPU (cms-patatrack#20)

* add `riemannFit` and `gpu` modifiers and workflows for Riemann fit and GPU modules
* switch GPU modules with `gpu` modifier

* Add a DQM sequence for pixel-only tracking (cms-patatrack#23)

* add a DQM sequence for pixel-only tracking
* add pixelTrackingOnlyDQM sequences to the pixelTrackingOnly workflows
* add pixelTrackingOnlyDQM sequences to the riemannFit and gpu workflows

* Synchronise with CMSSW_10_1_0_pre2

* Synchronise with CMSSW_10_1_0_pre3

* Synchronise with CMSSW_10_1_0

* Synchronise with CMSSW_10_2_0_pre1

* Synchronise with CMSSW_10_2_0_pre2

* Synchronise with CMSSW_10_2_0_pre3

* Synchronise with CMSSW_10_2_0_pre4

* Synchronise with CMSSW_10_2_0_pre5

* Port the Riemann fit to CUDA (cms-patatrack#60)

  - the CPU Riemann fit works using all combinations between the 2 booleans: `useErrors` and `useMultipleScattering`;
  - the standalone version of the GPU Riemann fit has been updated in order to explore all possibilities among the 2 booleans above: all of them work and produce identical results up to 1e-5 precision (the default one, 1e-6 fails when enabling multiScattering, most likely due to matrix inversions);
  - the GPU version of the Riemann fit within CMSSW works, with 1 fit assigned to each thread, with 32 threads/warps, all dynamically computed.

Things that needs a "hack":

  - limit the "dynamic" size of Eigen matrices to at most, 4x4, which is just fine for quadruplets. Using anything wider will cause errors which I *believe* is related to the stack size of threads on the GPU;
  - cast matrices to be inverted to 4x4 (was done before the previous point: will revert it back and see if that's still needed or not, but I believe it is); this was done in order to "specialize" the `invert()` call to something that is "natively" supported by Eigen on GPU (that brought in also few `__host__` `__device__` here and there in Eigen);
  - fix the alignment of the `struct` holding the results of the fit, since its size was different on GPU and CPU, causing an annoying off-by-one effect.

* Synchronise with CMSSW_10_2_0_pre6

* update relval input strings (102X)

* Synchronise with CMSSW_10_2_0

* adding NANO step to 2018 MC workflows

* Merge pull request #24024 from prebello/from-CMSSW_10_2_X_2018-07-24-2300

update relval input strings (102X)

* Synchronise with CMSSW_10_2_2_patch1

* activate the new SiStripCalSiStripCalBiasScan in the matrix test

* Add pixel tracking workflows for data (cms-patatrack#144)

* add HEM-15-16 relval workflows

* introduce RelVal for hybrid emulation (with ZS10 repacking),HLT-like zero-suppression (with normal ZS repacking),RECO of 2015 PbPb VirginRaw data

* add 3 production workflows for premix2018

* Merge pull request #24550 from zhenhu/add-HEM-15-16-relvals_102X

[10.2.X] add HEM-15-16 relval workflows

* Merge pull request #24618 from zhenhu/addPremixProdWfs_102X

backport [10.2.X] add 3 production workflows for premix2018

* Synchronise with CMSSW_10_2_5

* Synchronise with CMSSW_10_2_6

* Synchronise with CMSSW_10_4_0_pre2

* Synchronise with CMSSW_10_4_0_pre3

* Synchronise with CMSSW_10_4_0_pre4

* Synchronise with CMSSW_10_4_0

* Change GPU workflow numbering: .7->.51, .8->.52, .9->.53 (cms-patatrack#259)

* Synchronise with CMSSW_10_5_0_pre1

* Synchronise with CMSSW_10_5_0_pre2

* Synchronise with CMSSW_10_6_0_pre2

* Rework the Riemann fit and broken line fit (cms-patatrack#338)

Merge the Riemann and broken line fits into single configurable pixel
n-tuplet fitter, and extend it to work with up to 5 hits.

Mmake the broken line fit the default algorithm.

Try both triplets and quadruplets in the pixel "hole".

Limit pT used to compute the multple scattering.

Use the inline Cholesky decomposition.

Generic clean up and improvements.

* Synchronise with CMSSW_10_6_0

* Synchronise with CMSSW_11_0_0_pre7

* Fix broken merge in Configuration/PyReleaseValidation

* Synchronise with CMSSW_11_0_0_pre11

* Update and extend the Patatrack workflows

Update the definition of the Patatrack workflows to avoid conflicts in CMSSW_11_0_0_pre12:
  - renumber to avoid conflicts with #27392
  - reimplement the definition following #28281
  - add Z->mumu 2018 realistic workflows
  - add TTbar and Z->mumu 2021 realistic workflows
  - move the data workflows to 2018D, HLTPhysics and JetHT

* Synchronise with CMSSW_11_0_0_pre13

* Backport: add ECAL-only workflows for 2018 data (#30105)

* Add tracking-only and pixel tracking-only workflows for Run 3 (#30338)

* Backport: add ECAL-only and HCAL-only workflows for MC and data (#30350)

Backport #30105: add ECAL-only workflows for data.
Backport #30136: add HCAL-only workflows for MC and data.

* Synchronise with CMSSW_11_2_0_pre2

* Update the RelVal workflows and the CPU customisation (cms-patatrack#549)

Update the RelVal workflows and the CPU customisation:
  - change the .501 workflow to run the full Patatrack pixel track reconstruction on CPU
  - add a customisation to run the Patatrack reconstruction with triplets, on CPU and GPU
  - add the .505 and .506 workflows to reconstruct triplets, on CPU and GPU

Co-authored-by: Andrea Bocci <andrea.bocci@cern.ch>

* Synchronise with CMSSW_11_2_0_pre8

* Synchronise with CMSSW_11_2_0_pre9

* Synchronise with CMSSW_11_2_0_pre10

* Backport: add RecoFakeHLT steps to 2018 Patatrack WFs to enable GPU modifiers (#32333)

The 2018 GPU matrix WFs (10824.502, 10824.512, 10824.522, 10842.502) produced the same cmsDriver configurations as their CPU counterparts. This PR fixes this by setting the gpu modifier also for the RecoFakeHLT step.

* Synchronise with CMSSW_11_3_0_pre1

* Synchronise with CMSSW_11_3_X_2020-12-24-2300

* Document the new Pixel workflows

Co-authored-by: Matti Kortelainen <matti.kortelainen@cern.ch>
Co-authored-by: Marco Rovere <rovere@users.noreply.github.com>
Co-authored-by: Patricia <patricia.rebello.teles@cern.ch>
Co-authored-by: Francesco <Francesco.Fabozzi@cern.ch>
Co-authored-by: cmsbuild <cmsbuild@cern.ch>
Co-authored-by: mmusich <marco.musich@cern.ch>
Co-authored-by: zhen <zhen.hu@cern.ch>
Co-authored-by: Vincenzo Innocente <vincenzo.innocente@cern.ch>
Co-authored-by: Mariarosaria D'Alfonso <dalfonso@cern.ch>
Co-authored-by: AdrianoDee <AdrianoDee@users.noreply.github.com>
Co-authored-by: Thomas Reis <thomas.reis@cern.ch>
marco-link added a commit to marco-link/cmssw that referenced this pull request Apr 21, 2021
* Code check

* Add PPS geometry and simulation tags to GTs

* Changing the proton propagator to the Optical Function method

* Revert back to standard naming for Phase1 classes

* Fixed include guards

* Adding the new filter modules

* Code format

* Remove unused debug for-loop.

* Adding the multiple daughters of same kind possibility

* Use esConsumes for geometry and ETTmap records.

* Codecheck update

* Fix the deug statements

* use old hadronic physics

* Correcting the clang build err
	- corrected splelling in header guard

* fixed compilation

* add 14 TeV VBF H>inv pythia cfg

* added update on low-energy GFlash

* fix issues with clang tidy and static analyzer

* First skeleton of a Trackster from Simcluster producer

* Fix bug

* Add simtracksters into the event content

* Make SimTracksters part of TICL Tasks

* Create a SimTrackster per SimCluster

* Deleted methods should be public

* keep multiclusters from simtracksters

* cleanup

* add multiclusters from simtrackstrs

* add simtracksters to validation

* Make multiplicity take the simulated energy fraction

* add simTracksters iteration to validation

* replacing tabs with spaces and speeding up --help

* make enabling simclusters validation plots

* cleanup

* Add SimTracksters to FEVT and PreValidation Sequence

* Fix for premix wf

* get ES data through token

* update and reorder BuildFile

* adding missing dependencies in BuildFile

* Patatrack integration - Pixel workflows (12/N) (cms-sw#31854)

* Add workflows for Riemann fit and GPU (cms-patatrack#20)

* add `riemannFit` and `gpu` modifiers and workflows for Riemann fit and GPU modules
* switch GPU modules with `gpu` modifier

* Add a DQM sequence for pixel-only tracking (cms-patatrack#23)

* add a DQM sequence for pixel-only tracking
* add pixelTrackingOnlyDQM sequences to the pixelTrackingOnly workflows
* add pixelTrackingOnlyDQM sequences to the riemannFit and gpu workflows

* Synchronise with CMSSW_10_1_0_pre2

* Synchronise with CMSSW_10_1_0_pre3

* Synchronise with CMSSW_10_1_0

* Synchronise with CMSSW_10_2_0_pre1

* Synchronise with CMSSW_10_2_0_pre2

* Synchronise with CMSSW_10_2_0_pre3

* Synchronise with CMSSW_10_2_0_pre4

* Synchronise with CMSSW_10_2_0_pre5

* Port the Riemann fit to CUDA (cms-patatrack#60)

  - the CPU Riemann fit works using all combinations between the 2 booleans: `useErrors` and `useMultipleScattering`;
  - the standalone version of the GPU Riemann fit has been updated in order to explore all possibilities among the 2 booleans above: all of them work and produce identical results up to 1e-5 precision (the default one, 1e-6 fails when enabling multiScattering, most likely due to matrix inversions);
  - the GPU version of the Riemann fit within CMSSW works, with 1 fit assigned to each thread, with 32 threads/warps, all dynamically computed.

Things that needs a "hack":

  - limit the "dynamic" size of Eigen matrices to at most, 4x4, which is just fine for quadruplets. Using anything wider will cause errors which I *believe* is related to the stack size of threads on the GPU;
  - cast matrices to be inverted to 4x4 (was done before the previous point: will revert it back and see if that's still needed or not, but I believe it is); this was done in order to "specialize" the `invert()` call to something that is "natively" supported by Eigen on GPU (that brought in also few `__host__` `__device__` here and there in Eigen);
  - fix the alignment of the `struct` holding the results of the fit, since its size was different on GPU and CPU, causing an annoying off-by-one effect.

* Synchronise with CMSSW_10_2_0_pre6

* update relval input strings (102X)

* Synchronise with CMSSW_10_2_0

* adding NANO step to 2018 MC workflows

* Merge pull request cms-sw#24024 from prebello/from-CMSSW_10_2_X_2018-07-24-2300

update relval input strings (102X)

* Synchronise with CMSSW_10_2_2_patch1

* activate the new SiStripCalSiStripCalBiasScan in the matrix test

* Add pixel tracking workflows for data (cms-patatrack#144)

* add HEM-15-16 relval workflows

* introduce RelVal for hybrid emulation (with ZS10 repacking),HLT-like zero-suppression (with normal ZS repacking),RECO of 2015 PbPb VirginRaw data

* add 3 production workflows for premix2018

* Merge pull request cms-sw#24550 from zhenhu/add-HEM-15-16-relvals_102X

[10.2.X] add HEM-15-16 relval workflows

* Merge pull request cms-sw#24618 from zhenhu/addPremixProdWfs_102X

backport [10.2.X] add 3 production workflows for premix2018

* Synchronise with CMSSW_10_2_5

* Synchronise with CMSSW_10_2_6

* Synchronise with CMSSW_10_4_0_pre2

* Synchronise with CMSSW_10_4_0_pre3

* Synchronise with CMSSW_10_4_0_pre4

* Synchronise with CMSSW_10_4_0

* Change GPU workflow numbering: .7->.51, .8->.52, .9->.53 (cms-patatrack#259)

* Synchronise with CMSSW_10_5_0_pre1

* Synchronise with CMSSW_10_5_0_pre2

* Synchronise with CMSSW_10_6_0_pre2

* Rework the Riemann fit and broken line fit (cms-patatrack#338)

Merge the Riemann and broken line fits into single configurable pixel
n-tuplet fitter, and extend it to work with up to 5 hits.

Mmake the broken line fit the default algorithm.

Try both triplets and quadruplets in the pixel "hole".

Limit pT used to compute the multple scattering.

Use the inline Cholesky decomposition.

Generic clean up and improvements.

* Synchronise with CMSSW_10_6_0

* Synchronise with CMSSW_11_0_0_pre7

* Fix broken merge in Configuration/PyReleaseValidation

* Synchronise with CMSSW_11_0_0_pre11

* Update and extend the Patatrack workflows

Update the definition of the Patatrack workflows to avoid conflicts in CMSSW_11_0_0_pre12:
  - renumber to avoid conflicts with cms-sw#27392
  - reimplement the definition following cms-sw#28281
  - add Z->mumu 2018 realistic workflows
  - add TTbar and Z->mumu 2021 realistic workflows
  - move the data workflows to 2018D, HLTPhysics and JetHT

* Synchronise with CMSSW_11_0_0_pre13

* Backport: add ECAL-only workflows for 2018 data (cms-sw#30105)

* Add tracking-only and pixel tracking-only workflows for Run 3 (cms-sw#30338)

* Backport: add ECAL-only and HCAL-only workflows for MC and data (cms-sw#30350)

Backport cms-sw#30105: add ECAL-only workflows for data.
Backport cms-sw#30136: add HCAL-only workflows for MC and data.

* Synchronise with CMSSW_11_2_0_pre2

* Update the RelVal workflows and the CPU customisation (cms-patatrack#549)

Update the RelVal workflows and the CPU customisation:
  - change the .501 workflow to run the full Patatrack pixel track reconstruction on CPU
  - add a customisation to run the Patatrack reconstruction with triplets, on CPU and GPU
  - add the .505 and .506 workflows to reconstruct triplets, on CPU and GPU

Co-authored-by: Andrea Bocci <andrea.bocci@cern.ch>

* Synchronise with CMSSW_11_2_0_pre8

* Synchronise with CMSSW_11_2_0_pre9

* Synchronise with CMSSW_11_2_0_pre10

* Backport: add RecoFakeHLT steps to 2018 Patatrack WFs to enable GPU modifiers (cms-sw#32333)

The 2018 GPU matrix WFs (10824.502, 10824.512, 10824.522, 10842.502) produced the same cmsDriver configurations as their CPU counterparts. This PR fixes this by setting the gpu modifier also for the RecoFakeHLT step.

* Synchronise with CMSSW_11_3_0_pre1

* Synchronise with CMSSW_11_3_X_2020-12-24-2300

* Document the new Pixel workflows

Co-authored-by: Matti Kortelainen <matti.kortelainen@cern.ch>
Co-authored-by: Marco Rovere <rovere@users.noreply.github.com>
Co-authored-by: Patricia <patricia.rebello.teles@cern.ch>
Co-authored-by: Francesco <Francesco.Fabozzi@cern.ch>
Co-authored-by: cmsbuild <cmsbuild@cern.ch>
Co-authored-by: mmusich <marco.musich@cern.ch>
Co-authored-by: zhen <zhen.hu@cern.ch>
Co-authored-by: Vincenzo Innocente <vincenzo.innocente@cern.ch>
Co-authored-by: Mariarosaria D'Alfonso <dalfonso@cern.ch>
Co-authored-by: AdrianoDee <AdrianoDee@users.noreply.github.com>
Co-authored-by: Thomas Reis <thomas.reis@cern.ch>

* Remove CLCT BX and LCT BX comparisons

* add valueMap producer for the track distances

* add monitoring of closest track

* use CTPPSRun2Geometry customization to fix HLT test with Run2 data

* use the iEvent.getHandle syntax and make TrackDistanceValueMapProducer a global EDProducer

* Use 112X geometry in online data global tags

* Use 112X geometry in offline data global tags

* fix in CTPPSRun2Geometry cust funt

* fix in CTPPSRun2Geometry cust funt

* fix HLT addOnTest to use CTPPSRun2Geometry custom funct

* Disable the deug possibility

* add 14 tev gluino model

* Remove trackingMaterialGroups_ForPhaseII.xml from T25/T26 dict

* add comment to deprecate Multiclusters

* add name and date

* Modernize code and simplify

* add the possibility to save only the first nth closest track distances

* fix container size

* Add a new workflow with V15 version of HGCal Geometry

* Modify Run3 workflows with changed version of RP devices

* add spaces

* Change nodigis to hasdigis

* Hit counter for shower trigger

* Use constants from CSCConstants

* Use constants from CSCConstants

* Added ZpEE and ZpMM 14TeV fragments

* Use auto

* Migrate EcalTPCondAnalyzer to use esConsumes and replace couts with LogVerbatim.

* Move CSCConstants.h to DataFormats/CSCDigi

* Better document the wiregroups

* Delete particleFlowBadHcalPseudoCluster_cff.py

Was added by accident instead of RecoParticleFlow/PFClusterProducer/python/particleFlowBadHcalPseudoCluster_cff.py

* migrate DQM/TrackingMonitor to esConsumes

* Move plugin out of SimDataFormats

* Fix the memory error

* Code format

* First skeletton of the harvester

* Added skeletton for the worker

* Starting to populate a few monitors

* Making use of detid's naming export features

* Updated to the new PPS DataFormat accessors

* Stripping the histogram containers from worker

* Using rechits instead of digis

* Filling the condition on pixel tracks

* Full implementation of pixels tracks selection stage

* Corrected RP station and pixel tracks cuts retrieval

* Starting to integrate geometry

* Full usage of db-loaded geometry for channels booking, starting to fill the harvester

* Dropped CLHEP dependency

* Also storing uncalibrated ToT distribution, first attempt to fit the t-vs-tot profile in harvester

* Applied code-format

* Inserted PPSTimingCalib selection/paths/output sequences in their right place

* Delete ME file after copy

* Small cleanup, filtered stream renaming

* Reverted back filtered stream name

* Matched new folders hierarchy

* Updated paths

* Added Geometry/Records to TimingCalibration build

* Added an uncalibrated/calibrated collection of diamond information

* PPS track filter stripped from PCL worker

* Added a test sequence wrapping up all PCL operations

* Cleanup

* Per-pot track filter for PPS

* More C++-friendly tracks counting

* Replaced the track filter by the HLT-defined one

* Cleanup

* Cleanup of timing calibration worker sequence

* Keeping the diamond digis instead of rechits to perform uncalibrated rechits "reconstruction"

* Cleanup of harvester includes

* Using a smart pointer to hold x-profile used in calibration fit

* Ensure PoolDBService is available before processing

* Delegating DB service availability check at plugin constructor level

* Avoiding a couple of try/catch blocks through CTPPSDiamondDetId::check

* Filters rules retrieved by reference instead of copies

* Thread-safe geometry retrieval from ES

* Code-format

* Dropping the DB service private attribute to create it at constructor and for each end job

* Transition type set for geometry consumes declaration

* Thread-safe ES geometry condition retrieval for PCL harvester

* Update CalibPPS/TimingCalibration/python/ALCARECOPPSCalTrackBasedSel_Output_cff.py

Co-authored-by: Silvio Donato <silvio.donato@cern.ch>

* Update CalibPPS/TimingCalibration/python/ALCARECOPPSCalTrackBasedSel_cff.py

Co-authored-by: Silvio Donato <silvio.donato@cern.ch>

* Cloning instead of deep-copying, Task/Sequence redefinition of the filtering/worker modules

* Replaced back ','s with '*'s for seqALCARECOPPSCalTrackBasedSel Sequence

* Using a reco task instead of a sequence for the taskALCARECOPPSTimingCalib task definition

* PCL WF test is now 1041

* Apply changes from : scram build code-checks / code-format

* use references

* Correct validation code for MB of HCAL

* Code check

* Utilize a phase2 scenario for testing dd4hep scenarios

* Clarify number of strips and half-strips

* Tighten assert statements

* put distrip and cfeb calculation in functions

* Fix typo

* Check in more detail the content of teh geometry

* Code check

* Clean BuildFiles

* rename and small fixes

* CLANG test

* cleanup of Track analyzer moving double loop to dqm harvester

* remove commented lines

* remove unwanted includes and cout usage

* remove duplicate includes and cout

* resolve code check issues

* Move the macros to a separate directory and adopt for dd4hep

* Tryto fix

* change logic in folded occupancy client to allow running for cosmics;update cosmics config

* Code check

* update HI sequence

* apply coding style

* reorganization in configs

* Add step3 and 4 for TICL only

* Move check for pixel active area into RectangularMTDTopology, make it usable for BTL as well

* Fix post merge compilation errors

* Code format

* Fix indentation in EDAlias python dump

* Add a new scenario D84 same as D70 but with updated tracker T24

* updated physics to 10.7

* removed obsolete

* removed remaining obsolete

* complete update

* complete update

* clean-up custom physics

* code-checks

* code-checks

* update input file in fwliteWithPythonConfig_cfg.py

* Remove dead essignments from L1Trigger/CSCTrackFinder

* Suggestions by Andrea Perrotta

* Suggestions by Andrea Perrotta

* Modified version of EB numbering scheme code which will work with modified EB algorithm

* deal with fastsim issue raised in cms-sw#33430

* Code check

* Update python tests - py3 only

* Code format

* use python3 for all tests

* Minor changes in the configs removing uneeded loads in view of the new GT

* uncommented the verbosity line and changed the names

* fixed the names

* correct for gen fragment naming conventions

* remove old cfgs

Co-authored-by: Sunanda <sunanda.banerjee@cern.ch>
Co-authored-by: Helena <helena.malbouisson@gmail.com>
Co-authored-by: cmsbuild <cmsbuild@cern.ch>
Co-authored-by: Luiz Mundim <luiz.mundim@cern.ch>
Co-authored-by: Davide Valsecchi <davide.valsecchi@cern.ch>
Co-authored-by: Aravind <aravindsugunan@gmail.com>
Co-authored-by: Thomas Reis <thomas.reis@cern.ch>
Co-authored-by: Vladimir <Vladimir.Ivantchenko@cern.ch>
Co-authored-by: Vincenzo Innocente <vincenzo.innocente@cern.ch>
Co-authored-by: Silvio <silvio.donato@cern.ch>
Co-authored-by: Theodor Christian Herwig <2671-therwig@users.noreply.gitlab.cern.ch>
Co-authored-by: mmusich <marco.musich@cern.ch>
Co-authored-by: Felice <felice.pantaleo@cern.ch>
Co-authored-by: Andrea Bocci <andrea.bocci@cern.ch>
Co-authored-by: Matti Kortelainen <matti.kortelainen@cern.ch>
Co-authored-by: Marco Rovere <rovere@users.noreply.github.com>
Co-authored-by: Patricia <patricia.rebello.teles@cern.ch>
Co-authored-by: Francesco <Francesco.Fabozzi@cern.ch>
Co-authored-by: zhen <zhen.hu@cern.ch>
Co-authored-by: Mariarosaria D'Alfonso <dalfonso@cern.ch>
Co-authored-by: AdrianoDee <AdrianoDee@users.noreply.github.com>
Co-authored-by: Sven Dildick <sven.dildick@cern.ch>
Co-authored-by: Chris West <christopher.alan.west@gmail.com>
Co-authored-by: Adinda de Wit <adinda.maite.de.wit@cern.ch>
Co-authored-by: Andrea <perrotta@cern.ch>
Co-authored-by: namppl <kyungwook.nam@cern.ch>
Co-authored-by: laurenhay <lmhay@buffalo.edu>
Co-authored-by: Leonardo Cristella <leonardo.cristella@cern.ch>
Co-authored-by: Laurent Forthomme <laurent.forthomme@cern.ch>
Co-authored-by: Laurent Forthomme <lforthomme@protonmail.com>
Co-authored-by: Carlo Battilana <carlo.battilana@gmail.com>
Co-authored-by: Jonas Rembser <jonas.rembser@cern.ch>
Co-authored-by: Suvankar Roy Chowdhury <suvankar.roy.chowdhury@cern.ch>
Co-authored-by: Erica Brondolin <erica.brondolin@cern.ch>
Co-authored-by: Fabio Cossutti <fabio.cossutti@ts.infn.it>
Co-authored-by: Mircho Rodozov <mrodozov@cern.ch>
Co-authored-by: Mircho Nikolaev Rodozov <mrodozov@cmsdev25.cern.ch>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants