Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update L1 tracking for HL-LHC (replacement of #34028) #34215

Merged
merged 21 commits into from Jul 7, 2021

Conversation

skinnari
Copy link
Contributor

Important:

Must be coupled to, and tested with, cms-data/L1Trigger-TrackFindingTracklet#3

PR description:

This PR is a fixed version of #34028 (git mess-up by me ... apologies!)

This PR includes updates to the L1 tracking for HL-LHC:

  • use DTC emulation (L1Trigger/TrackerDTC), the step between the Tracker and the L1 tracking system, along with associated, necessary changes to the L1 tracking code
  • algorithm updates to match HLS-based version (i.e. firmware)
  • code cleanup (removal of obsolete code, general code improvements)

PR validation:

  • validated via L1 tracking performance studies on ttbar PU=200 samples.
  • DQM histogram will show many "failed" as they check for any differences, but DQM checks by experts are as expected.

if this PR is a backport please specify the original PR and why you need to backport that PR:

This is not a backport.

aehart and others added 20 commits June 22, 2021 19:27
* Various changes for outputting PurgeDuplicates test-vectors.

* Removed some magic numbers.

* Updated trackfitstr() to be more readable and include more comments.

* Removed more magic.
* Update max steps per TM period to match HLS

* Updated MC dataset
* add beam spot constraint flag to save new bcon values

* add check on scaled chi2 value after beam constraint

* get rid of code duplication

* include comment on kalmanAddBeamConstr_ boolean

* take out unecessary function
* Anders DTC merge with a gazillion manual fixes

* propagate buildfile fixes

* code formats

* fix cherrypick  mess up

* Remove LayerProjection class

* Further cleanup of tracklet projections interface

* Correct number of processing steps by changing < to <=

* Add missing include of algorithm

* Remove extra const

* Remove some commented out code

* Remove duplicate code

* Fix message logger and DTC Stub for consistency with hybrid configuration

* Fix problem with writing of input link memories

* Create Residual class that will replace LayerResidual and DiskResidual

* Remove the use of the class DiskResiduals

* Remove unused nMatch and nMatchDisk method of Tracklet

* (Re-)Implement the correction to writing the DTC data link file after moving functionality to Sector.

* Combine addMatch method for disk and layers into on method

* combine the disk and layer match into one method

* Remove some redundant poiters to l1tstubs

* Pass iSeed to Tracklet

* Introduce an InputRouter module. Does not change functionality, but simplifies/unifies the code and makes it similar to the HLS

* Cleanup of writing the DTC link files

* Change processing order such that all steps in one sector are done and then move to the next sector. This saves memory

* Interface updates for CMSSW following change to module processing order

* Change in VMRouter to processing PS links before 2S in disks

* Cleanup of unused iSector variable in processs and memory modules

* Fixes to make the HybridTracks_cfg.py run

* Cleanup of hardcoded numbers etc.

* Updates to MP to put all regions into one memory slot in the ProjectionTemp

* Fix to calculation of irinv for projections - no matches what is done in HLS

* Remove now unused file paths for the old cable mapping code

* Correct missplaced curly bracket

* Fixes for the displaced tracking

* Fix to avoid duplicate VMSTE name in D1 for standard configuration

* Address comments from Louise S.

* Ran scram b code-format

* Address comments from Louise S.

* Addressing more comments from Louise S.

* More fixes to comments

* Make running hybrid default (not displaced)

* restore buildfile

* Add DTC link config to Settings.h

* Changes to suppress warning in MatchCalculator when running displaced tracking

* Fix typo introduced in code cleanup for MatchProcessor

Co-authored-by: Anders Ryd <ar322@cornell.edu>
Co-authored-by: Anders <aryd@cern.ch>
* Fix to compiler warning and removal of hardoded value in MatchCalculator

* Updates to fix names of pins to make unique

Co-authored-by: Anders Ryd <ryd@Anderss-MacBook-Pro.local>
* Fix to compiler warning and removal of hardoded value in MatchCalculator

* Updates to fix names of pins to make unique

* Migrate to new format of table for phi correction memory

* Remove unused TETableBase class

* Cleanup of VMRouterTable

* Cleanup of LUT for bend consistency

* Cleanup of LUT used in MEU for bend consistency

* Cleanup code for disk bend table

* cleanup of code to generate table used by TE

* Cleanup of TP LUTs

* Cleanup of LUTs for MatchCalculator cuts

* Fix names of new LUT when writing to files

* Cleanup of writing LUTs for the combined modules

* Add code to write projection bend table for disk projections

* More cleanup to remove hard coded layer and seed numbers

* Cleanup of hardcoded numbers

* Remove few more hardcoded constants:

* Some further cleanup based on comments from Louise S.

Co-authored-by: Anders Ryd <ryd@Anderss-MacBook-Pro.local>
Co-authored-by: Louise Skinnari <louise.skinnari@cern.ch>
* Updates to MatchProcessor so that we can get full agreement with HLS version

* Fix to LUT for adding stub in VMStubsTEMemory when running extendted tracking

Co-authored-by: Anders Ryd <aryd@cern.ch>
…ing (#82)

* Corrections to fix updates to new LUTs that broke the displaced tracking

* Fix compile errors in CMSSW

* Remove commented out code

Co-authored-by: Louise Skinnari <louise.skinnari@cern.ch>
* fix matchtp variables

* remove d=0 variables

* switch values for cms constants

* change pi to constant

* change trk_d0 sign

* move tmp_tp_d0 and tmp_tp_z0 calc up

* flip tp_d0 sign
@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34215/23458

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 2, 2021

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ce158d/16398/summary.html
COMMIT: 986a71d
CMSSW: CMSSW_12_0_X_2021-07-01-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34215/16398/install.sh to create a dev area with all the needed externals and cmssw changes.

CMS StaticAnalyzer warnings: There are 4 EventSetupRecord::get warnings. See https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ce158d/16398/llvm-analysis/esrget-sa.txt for details.

RelVals-INPUT

  • 136.761DAS Error

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 38
  • DQMHistoTests: Total histograms compared: 2785711
  • DQMHistoTests: Total failures: 327
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2785362
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 37 files compared)
  • Checked 160 log files, 37 edm output root files, 38 DQM output files
  • TriggerResults: no differences found

@qliphy
Copy link
Contributor

qliphy commented Jul 2, 2021

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 2, 2021

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ce158d/16416/summary.html
COMMIT: 986a71d
CMSSW: CMSSW_12_0_X_2021-07-01-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34215/16416/install.sh to create a dev area with all the needed externals and cmssw changes.

CMS StaticAnalyzer warnings: There are 4 EventSetupRecord::get warnings. See https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ce158d/16416/llvm-analysis/esrget-sa.txt for details.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 10 differences found in the comparisons
  • DQMHistoTests: Total files compared: 38
  • DQMHistoTests: Total histograms compared: 2785711
  • DQMHistoTests: Total failures: 344
  • DQMHistoTests: Total nulls: 1
  • DQMHistoTests: Total successes: 2785344
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -0.004 KiB( 37 files compared)
  • DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
  • Checked 160 log files, 37 edm output root files, 38 DQM output files
  • TriggerResults: no differences found

@cecilecaillol
Copy link
Contributor

+l1

@srimanob
Copy link
Contributor

srimanob commented Jul 7, 2021

+Upgrade

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 7, 2021

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

@qliphy
Copy link
Contributor

qliphy commented Jul 7, 2021

+1

@qliphy
Copy link
Contributor

qliphy commented Jul 7, 2021

@skinnari with this PR and cms-data/L1Trigger-TrackFindingTracklet#3
we got many failures in IB, e.g.

https://cmssdt.cern.ch/SDT/cgi-bin/logreader/slc7_amd64_gcc900/CMSSW_12_0_X_2021-07-07-1100/pyRelValMatrixLogs/run/23234.103_TTbar_14TeV+2026D49Aging3000+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTrigger+RecoGlobal+HARVESTGlobal/step2_TTbar_14TeV+2026D49Aging3000+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTrigger+RecoGlobal+HARVESTGlobal.log#/

RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file L1Trigger/TrackFindingTracklet/data/dtclinklayerdisk.dat anywhere in the search path.

It seem you deleted dtclinklayerdisk.dat with cms-data/L1Trigger-TrackFindingTracklet#3
Can you have a quick look and fix? Thanks!

@qliphy
Copy link
Contributor

qliphy commented Jul 8, 2021

@skinnari

Looking again,

CMSSW_12_0_X_2021-07-07-1100 included
cms-sw/cmsdist#7109

while only from the next IB includes #34215
CMSSW_12_0_X_2021-07-07-1900

so we probably should wait for the new IB test to check the results.

@smuzaffar
Copy link
Contributor

@qliphy , the cmssw PR was merged after 11h00 CERN time that is why it was not picked up for 11h00 IB. In future when you need to merge changes which requires externals updates then please do 10-15 mins before the hour e.g. etither at 10h45 ( for 11h00 IB) or 22h45 (for 23h00 IB).

@skinnari
Copy link
Contributor Author

skinnari commented Jul 9, 2021

hi, just wanted to confirm that everything was indeed fine with the new IBs? please let me know otherwise, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

10 participants