Update L1 tracking for HL-LHC (fixed)

[skinnari]

PR description:

This PR is a fixed version of #33837 , made to master branch instead of CMSSW_11_3_X plus fixing "tests failed" (that were due to changes around handling of data files).

This PR includes updates to the L1 tracking for HL-LHC:

• use DTC emulation (L1Trigger/TrackerDTC), the step between the Tracker and the L1 tracking system, along with associated, necessary changes to the L1 tracking code
• algorithm updates to match HLS-based version (i.e. firmware)
• code cleanup (removal of obsolete code, general code improvements)

Note:

• this PR must be coupled with an associated update to: Update data-files (match cms-sw PR #34028) cms-data/L1Trigger-TrackFindingTracklet#3

PR validation:

Validated via L1 tracking performance studies on ttbar PU=200 samples.

if this PR is a backport please specify the original PR and why you need to backport that PR:

This is not a backport.

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34028/23158

• This PR adds an extra 992KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-checks:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34028/23158/code-checks.patch
  e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34028/23158/code-checks.patch | patch -p1
  You can also run scram build code-checks to apply code checks directly

• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34028/23158/code-format.patch
  e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34028/23158/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34028/23161

• This PR adds an extra 800KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34028/23161/code-format.patch
  e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34028/23161/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34028/23162

• This PR adds an extra 800KB to repository

[cmsbuild]

A new Pull Request was created by @skinnari (Louise Skinnari) for master.

It involves the following packages:

L1Trigger/TrackFindingTMTT
L1Trigger/TrackFindingTracklet
L1Trigger/TrackerDTC

@cmsbuild, @rekovic, @srimanob, @cecilecaillol, @kpedro88 can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @erikbutz this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[cecilecaillol]

please test

[skinnari]

ugh I am sorry, I had to rebase it since I had used a nightly when setting up that was no longer available, and it seems that it now committed that whole history. do i need to close and reopen again?

[skinnari]

I think I have to close this and reopen again. I am very sorry. I messed up git and not sure how to undo. For the future, is there a way to directly know which release one should use when making PRs? I had initially used a CMSSW_11_3_X version but had to make PR to master, and then picked the most recent nightly (which now was no longer available and I had to move to a different release)...

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34028/23453

• This PR adds an extra 556KB to repository

• Found files with invalid states:

  • RecoTracker/MkFit/plugins/MkFitHitConverter.cc:
    • Added: 9ce4863
    • Modified: c8b1553, 1127f54
    • Deleted: 78094ae

• There are other open Pull requests which might conflict with changes you have proposed:

  • File CalibTracker/SiStripChannelGain/BuildFile.xml modified in PR(s): BuildFiles cleaning #34191
  • File CalibTracker/SiStripESProducers/BuildFile.xml modified in PR(s): Fix SiStripQuality merging with FED errors from a DQM file #34082
  • File CalibTracker/SiStripESProducers/plugins/BuildFile.xml modified in PR(s): Fix SiStripQuality merging with FED errors from a DQM file #34082
  • File CalibTracker/SiStripESProducers/plugins/real/SiStripQualityESProducer.cc modified in PR(s): Fix SiStripQuality merging with FED errors from a DQM file #34082
  • File CalibTracker/SiStripESProducers/src/SiStripQualityHelpers.cc modified in PR(s): Fix SiStripQuality merging with FED errors from a DQM file #34082
  • File CondCore/Utilities/scripts/conddb modified in PR(s): Fix for conddb command line tool #34192, Move conddb to python3 #34151
  • File Configuration/AlCa/python/autoCond.py modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File Configuration/Geometry/python/dict2021Geometry.py modified in PR(s): Run 2, Run3, and Phase2 tracker: adding T27 (geometry with bricked pixels) and support for storing bricked pixel information #34120, Run3-gex79 Take P4 as the default PPS for Run3 scenario #34174
  • File Configuration/ProcessModifiers/python/trackingMkFit_cff.py modified in PR(s): Enable mkFit for initialStepPreSplitting with trackingMkFit modifier #34198
  • File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Add customisation to make sure that upgrade WFs will cross LS in the short matrix  #32957
  • File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Run 2, Run3, and Phase2 tracker: adding T27 (geometry with bricked pixels) and support for storing bricked pixel information #34120, Add run3 tracking low pu era #33532, Bianca pinolini ecal reco phase2 commit unimib #34212
  • File DQMOffline/CalibCalo/src/DQMSourcePi0.h modified in PR(s): DQM: migrate modules to esConsumes #34208
  • File DataFormats/PatCandidates/interface/PackedCandidate.h modified in PR(s): Look Up Table Approach for miniAOD packedCandidates #33622
  • File FWCore/Framework/src/EDConsumerBase.cc modified in PR(s): Implement persistence for ProcessBlock products #33969
  • File FWCore/Integration/test/BuildFile.xml modified in PR(s): Implement persistence for ProcessBlock products #33969
  • File GeneratorInterface/LHEInterface/plugins/ExternalLHEProducer.cc modified in PR(s): Refactoring gen weight storage in EDM + Nano integration #32167
  • File Geometry/ForwardCommonData/data/zdcmaterials/2021/v1/zdcmaterials.xml modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File Geometry/HGCalCommonData/interface/HGCalTypes.h modified in PR(s): Phase2-hgx287X Next step for V16 Geometry for HGCAL #34201
  • File Geometry/HGCalCommonData/src/HGCalWaferMask.cc modified in PR(s): Phase2-hgx287X Next step for V16 Geometry for HGCAL #34201
  • File HLTrigger/Configuration/python/HLT_FULL_cff.py modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File HLTrigger/Configuration/python/HLT_Fake1_cff.py modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File HLTrigger/Configuration/python/HLT_Fake2_cff.py modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File HLTrigger/Configuration/python/HLT_Fake_cff.py modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File HLTrigger/Configuration/python/HLT_GRun_cff.py modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File HLTrigger/Configuration/python/HLT_HIon_cff.py modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File HLTrigger/Configuration/python/HLT_PIon_cff.py modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File HLTrigger/Configuration/python/HLT_PRef_cff.py modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File HLTrigger/Configuration/test/OnLine_HLT_FULL.py modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File HLTrigger/Configuration/test/OnLine_HLT_Fake.py modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File HLTrigger/Configuration/test/OnLine_HLT_Fake1.py modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File HLTrigger/Configuration/test/OnLine_HLT_Fake2.py modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File HLTrigger/Configuration/test/OnLine_HLT_GRun.py modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File HLTrigger/Configuration/test/OnLine_HLT_HIon.py modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File HLTrigger/Configuration/test/OnLine_HLT_PIon.py modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File HLTrigger/Configuration/test/OnLine_HLT_PRef.py modified in PR(s): Bianca pinolini ecal reco phase2 commit unimib #34212
  • File L1Trigger/TrackFindingTracklet/interface/Settings.h modified in PR(s): Added a flag for enabling the application of the TED and TRE tables. #30847
  • File L1Trigger/TrackFindingTracklet/interface/Tracklet.h modified in PR(s): Added a flag for enabling the application of the TED and TRE tables. #30847
  • File L1Trigger/TrackFindingTracklet/interface/TrackletEngineDisplaced.h modified in PR(s): Added a flag for enabling the application of the TED and TRE tables. #30847
  • File L1Trigger/TrackFindingTracklet/plugins/L1FPGATrackProducer.cc modified in PR(s): Added a flag for enabling the application of the TED and TRE tables. #30847
  • File L1Trigger/TrackFindingTracklet/src/FitTrack.cc modified in PR(s): Added a flag for enabling the application of the TED and TRE tables. #30847
  • File L1Trigger/TrackFindingTracklet/src/MatchCalculator.cc modified in PR(s): Added a flag for enabling the application of the TED and TRE tables. #30847
  • File L1Trigger/TrackFindingTracklet/src/Tracklet.cc modified in PR(s): Added a flag for enabling the application of the TED and TRE tables. #30847
  • File L1Trigger/TrackFindingTracklet/src/TrackletEngineDisplaced.cc modified in PR(s): Added a flag for enabling the application of the TED and TRE tables. #30847
  • File L1Trigger/TrackFindingTracklet/src/TripletEngine.cc modified in PR(s): Added a flag for enabling the application of the TED and TRE tables. #30847
  • File OnlineDB/SiStripESSources/plugins/BuildFile.xml modified in PR(s): BuildFiles cleaning #34191
  • File PhysicsTools/NanoAOD/plugins/NanoAODOutputModule.cc modified in PR(s): Storing Generator Filter Information in NanoAOD #34169, Refactoring gen weight storage in EDM + Nano integration #32167
  • File PhysicsTools/PatAlgos/python/slimming/miniAOD_tools.py modified in PR(s): Unify vertex association between CHS and PUPPI #33885
  • File PhysicsTools/PatAlgos/python/slimming/slimming_cff.py modified in PR(s): Unify vertex association between CHS and PUPPI #33885
  • File RecoBTag/ImpactParameter/plugins/IPProducer.h modified in PR(s): [RECONSTRUCTION] code-checks/format; clean code to remove global functions #34140
  • File RecoLocalTracker/SiStripRecHitConverter/plugins/BuildFile.xml modified in PR(s): BuildFiles cleaning #34191
  • File RecoTracker/IterativeTracking/python/InitialStep_cff.py modified in PR(s): Enable mkFit for initialStepPreSplitting with trackingMkFit modifier #34198
  • File RecoTracker/MkFit/plugins/BuildFile.xml modified in PR(s): BuildFiles cleaning #34191
  • File RecoTracker/MkFit/plugins/MkFitOutputConverter.cc modified in PR(s): Enable mkFit for initialStepPreSplitting with trackingMkFit modifier #34198
  • File SimDataFormats/GeneratorProducts/interface/LHEEventProduct.h modified in PR(s): Refactoring gen weight storage in EDM + Nano integration #32167
  • File Validation/RecoEgamma/plugins/ElectronMcFakeValidator.cc modified in PR(s): Calotower -> HCAL rechit for EGM ID variables #33666
  • File Validation/RecoEgamma/plugins/ElectronMcSignalValidator.cc modified in PR(s): Calotower -> HCAL rechit for EGM ID variables #33666
  • File Validation/RecoTrack/python/plotting/trackingPlots.py modified in PR(s): Enable mkFit for initialStepPreSplitting with trackingMkFit modifier #34198, EXO regional tracking iteration #32806, shift legend's position in MTV tuning plots #34210

[cmsbuild]

Pull request #34028 was updated. @SiewYan, @andrius-k, @gouskos, @lveldere, @chayanit, @wajidalikhan, @sbein, @ianna, @kpedro88, @Martin-Grunewald, @rekovic, @tlampen, @alberto-sanchez, @pohsun, @santocch, @cecilecaillol, @perrotta, @civanch, @yuanchao, @makortel, @ErnestaP, @ahmad3213, @cmsbuild, @agrohsje, @fwyzard, @GurpreetSinghChahal, @smorovic, @davidlange6, @smuzaffar, @Dr15Jones, @jpata, @cvuosalo, @mdhildreth, @jfernan2, @slava77, @ggovi, @qliphy, @fabiocos, @francescobrivio, @malbouis, @ssekmen, @mkirsano, @jordan-martins, @kmaeshima, @emeschi, @alja, @srimanob, @silviodonato, @fgolf, @mariadalfonso, @rvenditti can you please check and sign again.

[kpedro88]

@skinnari "rebase" is a word with a precise meaning. it does not mean "merge". in the future, please follow http://cms-sw.github.io/tutorial-resolve-conflicts.html to rebase correctly.

[kpedro88]

to undo merges (or other mistakes) in git, there are several options:

1. try http://cms-sw.github.io/tutorial-resolve-conflicts.html#recovering-from-a-bad-rebase-or-merge
2. go back to your old working area in the previous release and push that state of your branch
3. use git reflog

[skinnari]

thanks @kpedro88, I think I have fixed it now, but because the instructions contains a "git push -f " it seems i cannot reopen the PR now?

[kpedro88]

Sometimes this happens when push -f is done after a PR is closed; I'm not sure why. I advise that, in the future, don't close PRs prematurely. (You may be able to push -f from the broken/merged branch if you still have it, reopen the PR, and then push -f from the non-merged branch again; but I leave it up to you whether you want to try this or just make a new PR.)

[makortel]

Sometimes this happens when push -f is done after a PR is closed; I'm not sure why.

Some explanation in isaacs/github#361

We're blocking the pull request reopen if the current head isn't a descendant of the stored head sha (which is what the head was when the pull request was closed). We are not allowing the reopen in that case, because there is no good way to tell what changes have happened while a pull request was closed and the head branch has changed.