GEM unpacker bug fix and update #30080

jshlee · 2020-06-02T17:01:21Z

PR description:

GEM unpacker was causing a crash for very high PU, due to large bx from large PU
Current GEMDigi bx is not calculated correctly
-- we currently use GEMDigi bx = GEB bx - VFAT bx, but this should always be 0
-- Changed to use GEMDigi bx = int(VFAT bx)
GEMRawToDigi module - to use same code for lightDQM (used by HW team)
link to talk at GEM DPG https://indico.cern.ch/event/924681/contributions/3885771/attachments/2049027/3433933/gemDPG_unpacker_20200602.pdf

PR validation:

validated with runTheMatrix.py 24811.0

if this PR is a backport please specify the original PR and why you need to backport that PR:

no backport needed yet

each bx is saved as new AMCdata

cmsbuild · 2020-06-02T17:01:50Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-06-02T17:06:38Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30080/15814

This PR adds an extra 28KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File EventFilter/GEMRawToDigi/interface/AMCdata.h modified in PR(s): [RFC] Test clang-tidy --checks performance-unnecessary-value-param #29933
- File EventFilter/GEMRawToDigi/interface/GEBdata.h modified in PR(s): [RFC] Test clang-tidy --checks performance-unnecessary-value-param #29933
- File EventFilter/GEMRawToDigi/plugins/GEMDigiToRawModule.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-use-auto #29936

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-checks:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30080/15814/code-checks.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30080/15814/code-checks.patch | patch -p1
You can also run scram build code-checks to apply code checks directly
code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30080/15814/code-format.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30080/15814/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2020-06-02T17:14:28Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-06-02T17:20:20Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30080/15815

This PR adds an extra 28KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File EventFilter/GEMRawToDigi/interface/AMCdata.h modified in PR(s): [RFC] Test clang-tidy --checks performance-unnecessary-value-param #29933
- File EventFilter/GEMRawToDigi/interface/GEBdata.h modified in PR(s): [RFC] Test clang-tidy --checks performance-unnecessary-value-param #29933
- File EventFilter/GEMRawToDigi/plugins/GEMDigiToRawModule.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-use-auto #29936

cmsbuild · 2020-06-02T17:20:46Z

A new Pull Request was created by @jshlee (Jason Lee) for master.

It involves the following packages:

EventFilter/GEMRawToDigi

@perrotta, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@Martin-Grunewald this is something you requested to watch as well.
@silviodonato, @dpiparo you are the release manager for this.

cms-bot commands are listed here

slava77 · 2020-06-03T21:32:47Z

@cmsbuild please test

cmsbuild · 2020-06-03T21:33:07Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/6790/console Started: 2020/06/03 23:38

cmsbuild · 2020-06-03T22:36:20Z

-1

Tested at: 627f3ee

CMSSW: CMSSW_11_2_X_2020-06-02-2300
SCRAM_ARCH: slc7_amd64_gcc820
You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-528447/6790/summary.html

I found follow errors while testing this PR

Failed tests: RelVals AddOn

RelVals:

When I ran the RelVals I found an error in the following workflows:
7.3 step3

runTheMatrix-results/7.3_CosmicsSPLoose_UP18+CosmicsSPLoose_UP18+DIGICOS_UP18+RECOCOS_UP18+ALCACOS_UP18+HARVESTCOS_UP18/step3_CosmicsSPLoose_UP18+CosmicsSPLoose_UP18+DIGICOS_UP18+RECOCOS_UP18+ALCACOS_UP18+HARVESTCOS_UP18.log

136.793 step3

runTheMatrix-results/136.793_RunDoubleEG2017C+RunDoubleEG2017C+HLTDR2_2017+RECODR2_2017reHLT_skimDoubleEG_Prompt+HARVEST2017/step3_RunDoubleEG2017C+RunDoubleEG2017C+HLTDR2_2017+RECODR2_2017reHLT_skimDoubleEG_Prompt+HARVEST2017.log

136.874 step3

runTheMatrix-results/136.874_RunEGamma2018C+RunEGamma2018C+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM/step3_RunEGamma2018C+RunEGamma2018C+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM.log

140.56 step2

runTheMatrix-results/140.56_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18/step2_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18.log

158.0 step3

runTheMatrix-results/158.0_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO/step3_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO.log

10042.0 step3

runTheMatrix-results/10042.0_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017/step3_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017.log

10024.0 step3

runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017.log

10824.0 step3

runTheMatrix-results/10824.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018.log

11634.0 step3

runTheMatrix-results/11634.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021/step3_TTbar_14TeV+TTbar_14TeV_TuneCP5_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021.log

12434.0 step3

runTheMatrix-results/12434.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2023_GenSimFull+DigiFull_2023+RecoFull_2023+HARVESTFull_2023+ALCAFull_2023/step3_TTbar_14TeV+TTbar_14TeV_TuneCP5_2023_GenSimFull+DigiFull_2023+RecoFull_2023+HARVESTFull_2023+ALCAFull_2023.log

10224.0 step3

runTheMatrix-results/10224.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU+NanoFull_2017PU/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU+NanoFull_2017PU.log

250202.181 step4

runTheMatrix-results/250202.181_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25/step4_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25.log

AddOn:

I found errors in the following addon tests:

cmsDriver.py RelVal -s HLT:2018,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_2018 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_2018_MC.root --fileout file:RelVal_Raw_2018_MC_HLT_RECO.root : FAILED - time: date Thu Jun 4 00:32:22 2020-date Thu Jun 4 00:19:47 2020 s - exit: 16640
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run3_data_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run3_pp_on_PbPb --processName=HLTRECO --filein file:RelVal_Raw_HIon_DATA.root --fileout file:RelVal_Raw_HIon_DATA_HLT_RECO.root : FAILED - time: date Thu Jun 4 00:26:18 2020-date Thu Jun 4 00:19:51 2020 s - exit: 16640
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run3_mc_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run3 --processName=HLTRECO --filein file:RelVal_Raw_GRun_MC.root --fileout file:RelVal_Raw_GRun_MC_HLT_RECO.root : FAILED - time: date Thu Jun 4 00:31:49 2020-date Thu Jun 4 00:19:53 2020 s - exit: 16640
cmsDriver.py RelVal -s HLT:2018,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_2018 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_2018_DATA.root --fileout file:RelVal_Raw_2018_DATA_HLT_RECO.root : FAILED - time: date Thu Jun 4 00:30:22 2020-date Thu Jun 4 00:19:59 2020 s - exit: 16640
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run3_mc_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run3 --processName=HLTRECO --filein file:RelVal_Raw_PRef_MC.root --fileout file:RelVal_Raw_PRef_MC_HLT_RECO.root : FAILED - time: date Thu Jun 4 00:26:48 2020-date Thu Jun 4 00:20:00 2020 s - exit: 16640
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run3_data_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run3 --processName=HLTRECO --filein file:RelVal_Raw_PIon_DATA.root --fileout file:RelVal_Raw_PIon_DATA_HLT_RECO.root : FAILED - time: date Thu Jun 4 00:25:44 2020-date Thu Jun 4 00:20:07 2020 s - exit: 16640
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run3_mc_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run3_pp_on_PbPb --processName=HLTRECO --filein file:RelVal_Raw_HIon_MC.root --fileout file:RelVal_Raw_HIon_MC_HLT_RECO.root : FAILED - time: date Thu Jun 4 00:27:39 2020-date Thu Jun 4 00:20:10 2020 s - exit: 16640
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run3_data_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run3 --processName=HLTRECO --filein file:RelVal_Raw_PRef_DATA.root --fileout file:RelVal_Raw_PRef_DATA_HLT_RECO.root : FAILED - time: date Thu Jun 4 00:26:06 2020-date Thu Jun 4 00:20:14 2020 s - exit: 16640
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run3_data_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run3 --processName=HLTRECO --filein file:RelVal_Raw_GRun_DATA.root --fileout file:RelVal_Raw_GRun_DATA_HLT_RECO.root : FAILED - time: date Thu Jun 4 00:32:55 2020-date Thu Jun 4 00:24:28 2020 s - exit: 16640
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run3_mc_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run3 --processName=HLTRECO --filein file:RelVal_Raw_PIon_MC.root --fileout file:RelVal_Raw_PIon_MC_HLT_RECO.root : FAILED - time: date Thu Jun 4 00:29:47 2020-date Thu Jun 4 00:25:17 2020 s - exit: 16640

cmsbuild · 2020-06-03T22:36:22Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

ggovi · 2020-06-25T14:44:48Z

+1

cmsbuild · 2020-06-25T14:45:14Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-06-26T08:36:54Z

+1
Since this PR change ths results obtained in 23234.1001 let me add @Martin-Grunewald @fwyzard @trtomei in CC

dildick · 2020-07-15T16:01:03Z

I must have missed this important PR. Does this affect the re-emulation of GEM-CSC trigger digis? If so, does it need to be back-ported to 11_1_X? @perrotta @jshlee

jshlee · 2020-07-17T05:56:40Z

@dildick yes, samples will need to be repacked.

rekovic · 2020-07-17T09:21:04Z

@dildick @jshlee
Do we need this PR for proper re-running L1T from the HLT TDR digi samples ?

dildick · 2020-07-17T13:45:25Z

@rekovic I don't know.

@jshlee This new GEM unpacker, will it work on HLT TDR samples? Will it also work for Run-3 Run3Winter20DRPremixMiniAOD samples? Please clarify.

jshlee · 2020-07-17T13:59:04Z

@dildick no, it will not work with previous releases unless it uses mapping from db.

dildick · 2020-07-17T14:04:41Z

So that will render a lot of samples useless. In my studies I need to use 11_2_X for CSC and GEM-CSC rate studies.

rekovic · 2020-07-17T14:52:58Z

@jshlee
Would it be possible to make a PR for 11_1_X with all the missing parts which would allow us to run on the HLT TDR samples ?

dildick · 2020-07-17T15:17:57Z

@rekovic Wouldn't that also interfere with Run-3 samples eventually? @jshlee I guess I need to know which stages of the MC need to be reprocessed so that GEM digs are correctly unpacked in 11_2_X

jshlee · 2020-07-17T15:32:56Z

@rekovic yes, i can backport if needed.
@dildick for run3, the mapping is in db, so this PR has no effect on it.

dildick · 2020-07-17T15:47:29Z

Right, but those samples were made in 11_0_X. With that information, would the samples still work?

jshlee · 2020-07-17T15:48:22Z

the samples for run3 should be fine.

jshlee · 2020-07-17T19:26:38Z

@dildick what is the sample that you're working with?
@rekovic what is the release for the HLT TDR samples? I didn't quite follow what was missing for it.

dildick · 2020-07-17T19:41:51Z

I'm using these: https://cmsweb.cern.ch/das/request?view=list&limit=50&instance=prod%2Fglobal&input=%2F*%2F*Run3*%2FGEN-SIM-RAW

fwyzard · 2020-07-17T22:42:38Z

The release for the HLT TDR is CMSSW_11_1_X - currently 11.1.2 IIRC - while the MC samples were originally produced with CMSSW_11_0_0.

jshlee · 2020-07-18T13:55:00Z

Thank you @fwyzard.
@rekovic since the mc samples are already produced, I'm not sure if there is any need for this PR to be backported. This is mainly a buy fix, and shouldn't change mc in anyway.

jshlee added 2 commits May 19, 2020 21:53

splitting to modules

c2712ab

updated GEM packer to use bx gemdigi map

046a2a6

each bx is saved as new AMCdata

cmsbuild added this to the CMSSW_11_2_X milestone Jun 2, 2020

cmsbuild added code-checks-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Jun 2, 2020

cmsbuild added code-checks-rejected and removed code-checks-pending labels Jun 2, 2020

code-checks-format

627f3ee

cmsbuild added code-checks-pending and removed code-checks-rejected labels Jun 2, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 2, 2020

cmsbuild added tests-started and removed tests-pending labels Jun 3, 2020

cmsbuild added comparison-notrun and removed comparison-pending tests-started labels Jun 3, 2020

cmsbuild added dqm-approved and removed dqm-pending labels Jun 24, 2020

cmsbuild added db-approved fully-signed and removed db-pending pending-signatures labels Jun 25, 2020

cmsbuild added orp-approved and removed orp-pending labels Jun 26, 2020

cmsbuild merged commit 42dc49c into cms-sw:master Jun 26, 2020

cmsbuild mentioned this pull request Jun 26, 2020

Strip clusterizer speedup: technical changes from #28304 #30046

Merged

seungjin-yang mentioned this pull request Jun 30, 2020

Re-add modules for GEM offline DQM #30468

Merged

watson-ij mentioned this pull request Sep 15, 2020

Update dummy emap to include layers 3-6 of GE0 #31374

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GEM unpacker bug fix and update #30080

GEM unpacker bug fix and update #30080

jshlee commented Jun 2, 2020

cmsbuild commented Jun 2, 2020

cmsbuild commented Jun 2, 2020

cmsbuild commented Jun 2, 2020

cmsbuild commented Jun 2, 2020

cmsbuild commented Jun 2, 2020

slava77 commented Jun 3, 2020

cmsbuild commented Jun 3, 2020 •

edited

cmsbuild commented Jun 3, 2020

cmsbuild commented Jun 3, 2020

ggovi commented Jun 25, 2020

cmsbuild commented Jun 25, 2020

silviodonato commented Jun 26, 2020

dildick commented Jul 15, 2020 •

edited

jshlee commented Jul 17, 2020

rekovic commented Jul 17, 2020

dildick commented Jul 17, 2020

jshlee commented Jul 17, 2020

dildick commented Jul 17, 2020

rekovic commented Jul 17, 2020

dildick commented Jul 17, 2020

jshlee commented Jul 17, 2020

dildick commented Jul 17, 2020

jshlee commented Jul 17, 2020

jshlee commented Jul 17, 2020

dildick commented Jul 17, 2020

fwyzard commented Jul 17, 2020

jshlee commented Jul 18, 2020

GEM unpacker bug fix and update #30080

GEM unpacker bug fix and update #30080

Conversation

jshlee commented Jun 2, 2020

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Jun 2, 2020

cmsbuild commented Jun 2, 2020

cmsbuild commented Jun 2, 2020

cmsbuild commented Jun 2, 2020

cmsbuild commented Jun 2, 2020

slava77 commented Jun 3, 2020

cmsbuild commented Jun 3, 2020 • edited

cmsbuild commented Jun 3, 2020

cmsbuild commented Jun 3, 2020

ggovi commented Jun 25, 2020

cmsbuild commented Jun 25, 2020

silviodonato commented Jun 26, 2020

dildick commented Jul 15, 2020 • edited

jshlee commented Jul 17, 2020

rekovic commented Jul 17, 2020

dildick commented Jul 17, 2020

jshlee commented Jul 17, 2020

dildick commented Jul 17, 2020

rekovic commented Jul 17, 2020

dildick commented Jul 17, 2020

jshlee commented Jul 17, 2020

dildick commented Jul 17, 2020

jshlee commented Jul 17, 2020

jshlee commented Jul 17, 2020

dildick commented Jul 17, 2020

fwyzard commented Jul 17, 2020

jshlee commented Jul 18, 2020

cmsbuild commented Jun 3, 2020 •

edited

dildick commented Jul 15, 2020 •

edited