Update of GEM & CSC trigger primitive emulators #38373

giovanni-mocellin · 2022-06-15T09:00:21Z

PR description:

This PR introduces major changes in the L1 trigger primitive emulators for GEM and GEM-CSC, plus some minor changes for CSC for Run-3 mode. In addition, it ensures the full compatibility of unpackers and emulators to the b904 lab test setups, utilized during the SW and FW validation.

Changes for GEM trigger primitive emulator:

GEM unpacker modified to add compatibility with GEM chambers in b904 lab setup mapping: added new file to CondDB and corresponding tags in test python config script, specified correct FED ID numbers for it
GEM trigger primitive emulator streamlined, with specific GE1/1 and GE2/1 parameters. Less mishmash between different chamber types: easier maintainability and flexibility
Updated to latest FW version, which includes: 4 clustering partitions (2 eta partitions each), and a maximum of 4 clusters built per clustering partition. Kept the maximum of 8 clusters selected and sent per chamber, and the maximum cluster size of 8 pads

Changes for CSC trigger primitive emulator:

CSC unpacker modified to add compatibility with CSC chambers in b904 lab setup mapping. Manual flags added to specify the type of chamber in use: B904ME11PositiveEndcap, B904ME11NegativeEndcap, B904ME234s2
CSC unpacker modified to add printouts (commented out by default)
Added new flag for run-3 CSC trigger primitive “run3”. It regulates the primitive quality data format. Earlier the quality format was linked to the CCLUT algorithm activation, now it’s independent. Needed for CSC outer rings, where CCLUT algorithm is not implemented (and will not be implemented) in FW, but the quality assignment follows run-3 format
CSC CLCT ordering between first and second has been updated to the one implemented in FW: first sort by quality, then by pattern, and finally by half strip number

Changes for GEM-CSC trigger primitive emulator:

Updated names and paths for GEM-CSC LUT files stored in cms-data (dedicated parallel PR done, more comments below in NOTE 1)
Modified the BX assigned to GEM trigger primitives readout through CSC DAQ: before it was referenced to CLCTs, now it is referenced to ALCTs. Thus, the time assignment takes into consideration the variable “alctMatchTime”. This entailed a modification to the CSC unpacker and packer
CSC unpacker modified to add compatibility with GEM chambers in b904 lab setup mapping. Manual flags added to specify the type of chamber in use: B904GE11Long, B904GE11Short
Major update to the GEM-CSC trigger primitive matching algorithm. A few more details below in NOTE 2

NOTE 1)
This PR is intrinsically linked to the PR #13 to L1Trigger-CSCTriggerPrimitives, which updates the lookup tables for the GEM-CSC trigger to the ones currently used in FW

NOTE 2)
Update to the GEM-CSC trigger primitive matching algorithm. Before it reflected an old concept of FW implementation, now it adheres completely to the FW implemented and deployed
A few notable changes:

Full independence of GEM-CSC and CSC-only emulators + streamlined GEM-CSC emulator implementation: much easier maintainability now
Added dedicated matching algorithms for the two GEM chamber layers
GEM primitives matching windows are updated to final applied values
GEM primitives matching in time is ALCT-centric. Selected GEM BX is the closest to CSC ALCT BX in which a GEM hit is found. Using same GEM-CSC BX difference preferences as in FW
CSC CLCT propagation to GEM (via LUTs) is the default matching algorithm
CSC LCT slope and run2pattern are amended by GEM trigger primitive position wrt CSC trigger primitive position by default
Update of quality assignment: swapped quality assignment between CLCT-2GEM and ALCT-2GEM, to conform to FW

NOTE 3)
In agreement with EMTF experts (@eyigitba), the flags that control the run-3 algorithms in CSC (and EMTF) have been kept off for this PR. Another PR (or a new commit) will come within a few days to abilitate the necessary flags. Since the PR includes changes in the run-3 algorithms, the tests were performed turning on such flags.
=> UPDATE => Flags were set to True with a new commit

PR validation:

The code has been tested and validated by re-emulating b904 lab and P5 data taken during the FW commissioning phase. Plots of comparisons between re-emulation and data show very good agreement. A couple of re-emulation files for a GE1/1-ME1/1 run in b904 and a ME4/2 run in b904 have been attached to this PR.
ReEmuFiles.zip

Note: the tests were done with run-3 flags activated, in order to verify the proper functioning of the implemented algorithms. As described before, now these flags have been turned off, and will be turned on in a second PR (or new commit), to come within a few days.
=> UPDATE => Flags were set to True with a new commit

cmsbuild · 2022-06-15T09:06:55Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38373/30565

This PR adds an extra 420KB to repository
Found files with invalid states:
- EventFilter/GEMRawToDigi/test/GEMeMap_GE11_b904_Odd.db:
  - Added: 70ce048
  - Deleted: 9510eae
- EventFilter/GEMRawToDigi/test/GEMeMap_GE11_b904_Even.db:
  - Added: 70ce048
  - Deleted: 9510eae

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38373/30565/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38373/30565/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

tvami · 2022-06-15T09:19:12Z

test parameters:

pull_request = Uodated GEM-CSC triager LUTs cms-data/L1Trigger-CSCTriggerPrimitives#13

cmsbuild · 2022-06-15T09:20:57Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38373/30566

This PR adds an extra 420KB to repository
Found files with invalid states:
- EventFilter/GEMRawToDigi/test/GEMeMap_GE11_b904_Odd.db:
  - Added: 70ce048
  - Deleted: 9510eae
- EventFilter/GEMRawToDigi/test/GEMeMap_GE11_b904_Even.db:
  - Added: 70ce048
  - Deleted: 9510eae

cmsbuild · 2022-06-15T09:21:23Z

A new Pull Request was created by @giovanni-mocellin for master.

It involves the following packages:

CalibMuon/CSCCalibration (alca)
CondFormats/CSCObjects (db, alca)
DQM/CSCMonitorModule (dqm)
DQM/L1TMonitor (dqm)
DataFormats/GEMDigi (upgrade, simulation)
EventFilter/CSCRawToDigi (reconstruction)
EventFilter/GEMRawToDigi (reconstruction)
L1Trigger/CSCTriggerPrimitives (l1)
L1Trigger/L1TGEM (l1)

@rekovic, @jpata, @cecilecaillol, @civanch, @yuanchao, @ahmad3213, @cmsbuild, @pmandrik, @epalencia, @emanueleusai, @mdhildreth, @AdrianoDee, @jfernan2, @slava77, @ggovi, @micsucmed, @francescobrivio, @malbouis, @clacaputo, @srimanob, @rvenditti, @tvami can you please review it and eventually sign? Thanks.
@valuev, @barvic, @jshlee, @tocheng, @watson-ij, @eyigitba, @Martin-Grunewald, @missirol, @rovere, @trtomei, @mmusich, @ptcox, @beaucero, @seemasharmafnal this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

tvami · 2022-06-15T09:23:11Z

Hi @giovanni-mocellin
I have several comments/questions

first a trivial one, you need to do code-checks (edit: you did this while I wrote the comment, thanks)
then please squash the commits to something more sensible, or a single commit, having 60(+1) commits changin 49 files is not ideal
then some real comments: why arent these conditions passed through GT instead of having to modify files in the cms-data?
I understand from the description that with the current version of the PR there is no way to test this, so makes we wonder what's the incentive to submit the PR before the commits that change the given flags?
Did you mean to use this in the 12_4_0 MC campaign? That seems too late

giovanni-mocellin · 2022-06-15T09:45:29Z

I'm sorry, it's my first PR to central CMSSW, so I'll ask probably trivial questions for probably trivial doubts...
Point by point:

Code-checks: OK
I totally understand that the changes are large, and mostly entangled. Should I do anything about it? I promise for next PR that I will be more graceful ;)
Could you please elaborate more about "these conditions passed through GT"? I'm modifying the LUTs in cms-data, since the old ones are not valid anymore - and will never be. The CSC firmware implementation utilizes only the new ones
I added some files that show the correct functioning of the new emulators - I get the fact that it may look not enough. So, we have two options: add a commit to activate the flags to this PR or go for another one later (as per description of this PR). EMTF experts wanted to have the flags of CSC and EMTF turned ON at once in a PR - it can also be this one, if you prefer
We don't aim for it. From L1 DPG: they don't want to push the Run-3 changes without checking them more throughly in MC

tvami · 2022-06-15T10:04:13Z

I totally understand that the changes are large, and mostly entangled. Should I do anything about it?

No worries, you can still fix it, maybe we can chat on mattermost on what do do exaclty

Could you please elaborate more about "these conditions passed through GT"?

Maybe @ptcox could chime in here from the CSC side? My proposal is to store these in an sqlite file instead of your 112 txt files in cms-data, and read it through that database, making sure there is a reproducibility for the past (*) and easy updates for the future

(*) this actually triggered me a new question: if you delete the files in cms-data, does that mean the Run-2 reconstruction wont be able to be reproduced?

add a commit to activate the flags to this PR

I'd actually prefer to have the full changes tested in the Jenkins here, meaning that if there is no objection to do it in this PR, I'd like you do commit the flag changes here.

We don't aim for it. From L1 DPG: they don't want to push the Run-3 changes without checking them more throughly in MC

That is reassuring, thanks!

tvami · 2022-06-15T10:10:52Z

EventFilter/GEMRawToDigi/test/runGEMUnpacker_cfg.py

+    process.GlobalTag.toGet = cms.VPSet(
+            cms.PSet(record = cms.string("GEMChMapRcd"),
+                     tag = cms.string("GEMChMapRcd"),
+                     connect = cms.string("sqlite_fip:EventFilter/GEMRawToDigi/test/GEMeMap_GE11_b904.db")


this is under test, so maybe it was just something temporary, but we just updated the GEM ChMap in the GT, how is this tag different? @hyunyong maybe?

This is for testing the trigger features in the b904 laboratory, where we have cosmic stands that require dedicated mapping files. This map file is indeed in line with the new GEM ChMap format

This file is utilized for a cosmic stand in b904 laboratory, where we validate the FW features. The mapping follows the format of the recently introduced GEM ChMap in the GT. Hope this answers the question, otherwise maybe @yeckang could know more.

@tvami
This unpacker is not for the P5. The data should come from the local setup in B904. And as you can expect, the mapping would be different from the one that we are using for the P5.

Ok that's good to know. But then this tag should be uploaded to the DB as a single-iov tag, with an appropriate naming, and then ES prefer that. tagging @hyunyong again for doing the upload

giovanni-mocellin · 2022-06-15T10:12:00Z

maybe we can chat on mattermost on what do do exaclty

Sure, thanks for your help!

if you delete the files in cms-data, does that mean the Run-2 reconstruction wont be able to be reproduced?
there is a reproducibility for the past

We only modified tables that were not used in Run-2, since both GEM-CSC features and CSC consistency were not used (in fact, CSC consistency will not be used in Run-3 either, but we wanted to keep the feature there)

I'd like you do commit the flag changes here

Will do

clacaputo · 2022-07-20T15:05:05Z

+reconstruction

resign, Update of GEM & CSC trigger primitive emulators #38373 (comment)

srimanob · 2022-07-20T15:08:59Z

+Upgrade

Re-sign. Last commit does not change Upgrade part.

perrotta · 2022-07-21T06:54:21Z

@cms-sw/dqm-l2 @cms-sw/simulation-l2 changes since your latest signature are rather limited: could you please review, and sign if you are fine with it?

emanueleusai · 2022-07-21T06:55:03Z

+1

tvami · 2022-07-21T19:05:34Z

@cms-sw/orp-l2 can we bypass the simulation signature? I think Vladimir did sign this already in the past so it should be fine

tvami · 2022-07-21T19:07:27Z

I think Vladimir did sign this already in the past so it should be fine

see #38373 (comment)
since then the only changes were that @perrotta requested

civanch · 2022-07-21T22:06:11Z

+1

cmsbuild · 2022-07-21T22:06:33Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-07-22T05:36:47Z

+1

ptcox · 2022-10-11T08:47:15Z

I'm certainly not making a claim like that! Can "track packing in miniAOD" depend on reco'ed muons? The only CSC information in even AOD, is AFAIK, only that associated with reco'ed muons (i.e. CSC rechits and segmnets contributing to reco'ed muons). Tim

…

Marco Musich ***@***.***> July 4, 2022 at 16:46 I still don't understand how the CSC readout can affect the track packing in miniAOD, but it's not up to me to sign this PR :) — Reply to this email directly, view it on GitHub <#38373 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGYLHXMO4VILDALVNDOB33VSL2LDANCNFSM5Y2QBMRA>. You are receiving this because you were mentioned.Message ID: ***@***.***>

cmsbuild added this to the CMSSW_12_5_X milestone Jun 15, 2022

cmsbuild added alca-pending code-checks-pending db-pending dqm-pending l1-pending orp-pending pending-signatures reconstruction-pending simulation-pending tests-pending upgrade-pending labels Jun 15, 2022

cmsbuild added code-checks-rejected code-checks-pending and removed code-checks-pending code-checks-rejected labels Jun 15, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 15, 2022

cmsbuild added the requires-external label Jun 15, 2022

tvami reviewed Jun 15, 2022

View reviewed changes

cmsbuild added code-checks-pending and removed code-checks-approved labels Jun 15, 2022

cmsbuild added alca-approved db-approved and removed alca-pending db-pending labels Jul 20, 2022

eyigitba mentioned this pull request Jul 20, 2022

[12_4_X] Adding extra protection against invalid data to EMTF PrimitiveSelection #38794

Merged

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Jul 20, 2022

cmsbuild added upgrade-approved and removed upgrade-pending labels Jul 20, 2022

cmsbuild added dqm-approved and removed dqm-pending labels Jul 21, 2022

cmsbuild added fully-signed simulation-approved and removed simulation-pending pending-signatures labels Jul 21, 2022

cmsbuild added orp-approved and removed orp-pending labels Jul 22, 2022

cmsbuild merged commit 106b43f into cms-sw:master Jul 22, 2022

This was referenced Jul 22, 2022

Add wfs to test Run3 skims #38830

Merged

throw in IBooker::bookME when illegal characters for the DQM GUI upload are encountered #38831

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update of GEM & CSC trigger primitive emulators #38373

Update of GEM & CSC trigger primitive emulators #38373

giovanni-mocellin commented Jun 15, 2022 •

edited

cmsbuild commented Jun 15, 2022

tvami commented Jun 15, 2022 •

edited

cmsbuild commented Jun 15, 2022

cmsbuild commented Jun 15, 2022

tvami commented Jun 15, 2022 •

edited

giovanni-mocellin commented Jun 15, 2022 •

edited

tvami commented Jun 15, 2022

tvami Jun 15, 2022

giovanni-mocellin Jun 15, 2022

giovanni-mocellin Jun 15, 2022

yeckang Jun 15, 2022

tvami Jun 15, 2022

giovanni-mocellin commented Jun 15, 2022 •

edited

clacaputo commented Jul 20, 2022

srimanob commented Jul 20, 2022

perrotta commented Jul 21, 2022

emanueleusai commented Jul 21, 2022

tvami commented Jul 21, 2022

tvami commented Jul 21, 2022

civanch commented Jul 21, 2022

cmsbuild commented Jul 21, 2022

perrotta commented Jul 22, 2022

ptcox commented Oct 11, 2022 via email

Update of GEM & CSC trigger primitive emulators #38373

Update of GEM & CSC trigger primitive emulators #38373

Conversation

giovanni-mocellin commented Jun 15, 2022 • edited

PR description:

PR validation:

cmsbuild commented Jun 15, 2022

tvami commented Jun 15, 2022 • edited

cmsbuild commented Jun 15, 2022

cmsbuild commented Jun 15, 2022

tvami commented Jun 15, 2022 • edited

giovanni-mocellin commented Jun 15, 2022 • edited

tvami commented Jun 15, 2022

tvami Jun 15, 2022

Choose a reason for hiding this comment

giovanni-mocellin Jun 15, 2022

Choose a reason for hiding this comment

giovanni-mocellin Jun 15, 2022

Choose a reason for hiding this comment

yeckang Jun 15, 2022

Choose a reason for hiding this comment

tvami Jun 15, 2022

Choose a reason for hiding this comment

giovanni-mocellin commented Jun 15, 2022 • edited

clacaputo commented Jul 20, 2022

srimanob commented Jul 20, 2022

perrotta commented Jul 21, 2022

emanueleusai commented Jul 21, 2022

tvami commented Jul 21, 2022

tvami commented Jul 21, 2022

civanch commented Jul 21, 2022

cmsbuild commented Jul 21, 2022

perrotta commented Jul 22, 2022

ptcox commented Oct 11, 2022 via email

giovanni-mocellin commented Jun 15, 2022 •

edited

tvami commented Jun 15, 2022 •

edited

tvami commented Jun 15, 2022 •

edited

giovanni-mocellin commented Jun 15, 2022 •

edited

giovanni-mocellin commented Jun 15, 2022 •

edited