Restore "PPS Digi integration into cmssw (final)" #32003

silviodonato · 2020-11-02T15:01:53Z

PR description:

PPS digi integration into cmssw. This is the last step towards the integration of PPS simulation into CMSSW and was delayed waiting for the new cms extended geometry (including PPS) to show up in the DB.

Original PR: #31943
Reverted with #31997
because of issue #31991

cmsbuild · 2020-11-02T15:02:12Z

The code-checks are being triggered in jenkins.

silviodonato · 2020-11-02T15:02:28Z

cc @mundim

silviodonato · 2020-11-02T15:02:37Z

assign simulation

cmsbuild · 2020-11-02T15:07:03Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32003/19518

This PR adds an extra 8KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/StandardSequences/python/DigiToRaw_cff.py modified in PR(s): Ecal phase2 new digitization 11 2 x #31726

silviodonato · 2020-11-02T15:07:18Z

test parameters:

workflows = 8.2, 11634.0

cmsbuild · 2020-11-02T15:07:22Z

New categories assigned: simulation

@mdhildreth,@civanch you have been requested to review this Pull request/Issue and eventually sign? Thanks

cmsbuild · 2020-11-02T15:07:23Z

A new Pull Request was created by @silviodonato (Silvio Donato) for master.

It involves the following packages:

Configuration/StandardSequences

@civanch, @silviodonato, @mdhildreth, @cmsbuild, @franzoni, @qliphy, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@fabiocos, @makortel, @felicepantaleo, @GiacomoSguazzoni, @rovere, @VinInn, @Martin-Grunewald, @lecriste, @mtosi, @dgulhan, @slomeo this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

silviodonato · 2020-11-02T15:07:25Z

please test

cmsbuild · 2020-11-02T15:07:37Z

The tests are being triggered in jenkins.
Test Parameters:

MATRIX_EXTRAS = 8.2,11634.0
CMSSW_11_2_X_2020-11-01-2300/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10440/console Started: 2020/11/02 16:08

cmsbuild · 2020-11-02T16:36:34Z

-1

Tested at: 2b6266a

CMSSW: CMSSW_11_2_X_2020-11-01-2300
SCRAM_ARCH: slc7_amd64_gcc820

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

add ZDC Digis to HI miniAOD #31971 @mandrenguyen: add ZDC Digis to HI miniAOD
Fix bug in NanoGEN precision setting #31973 @kdlong: Fix bug in NanoGEN precision setting
Removed cout statements from BMTF packer #31969 @dinyar: Removed cout statements from BMTF packer
UL scale and smearing for nanoAOD photons and electrons (2017 and 2018) #31974 @jainshilpi: UL scale and smearing for nanoAOD photons and electrons (2017 and 2018)

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-472d82/10440/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-472d82/10440/git-merge-result

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-472d82/10440/summary.html

I found follow errors while testing this PR

Failed tests: RelVals

RelVals:

When I ran the RelVals I found an error in the following workflows:
8.2 step2

runTheMatrix-results/8.2_BeamHalo_UP21+BeamHalo_UP21+DIGICOS_UP21+RECOCOS_UP21+ALCABH_UP21+HARVESTCOS_UP21/step2_BeamHalo_UP21+BeamHalo_UP21+DIGICOS_UP21+RECOCOS_UP21+ALCABH_UP21+HARVESTCOS_UP21.log

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

add ZDC Digis to HI miniAOD #31971 @mandrenguyen: add ZDC Digis to HI miniAOD
Fix bug in NanoGEN precision setting #31973 @kdlong: Fix bug in NanoGEN precision setting
Removed cout statements from BMTF packer #31969 @dinyar: Removed cout statements from BMTF packer
UL scale and smearing for nanoAOD photons and electrons (2017 and 2018) #31974 @jainshilpi: UL scale and smearing for nanoAOD photons and electrons (2017 and 2018)

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-472d82/10440/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-472d82/10440/git-merge-result

cmsbuild · 2020-11-02T16:36:36Z

Comparison job queued.

mundim · 2020-11-03T17:18:29Z

When I ran the RelVals I found an error in the following workflows:
8.2 step2

runTheMatrix-results/8.2_BeamHalo_UP21+BeamHalo_UP21+DIGICOS_UP21+RECOCOS_UP21+ALCABH_UP21+HARVESTCOS_UP21/step2_BeamHalo_UP21+BeamHalo_UP21+DIGICOS_UP21+RECOCOS_UP21+ALCABH_UP21+HARVESTCOS_UP21.log

I tried to figure out what is going on and what I saw is that the configuration for cosmic has a special digi config file and PPS is not included. What I can do it to issue another PR including PPS digi into the cosmic one, if that is ok with you all.

Thanks.

silviodonato · 2020-11-06T10:16:12Z

When I ran the RelVals I found an error in the following workflows:
8.2 step2
runTheMatrix-results/8.2_BeamHalo_UP21+BeamHalo_UP21+DIGICOS_UP21+RECOCOS_UP21+ALCABH_UP21+HARVESTCOS_UP21/step2_BeamHalo_UP21+BeamHalo_UP21+DIGICOS_UP21+RECOCOS_UP21+ALCABH_UP21+HARVESTCOS_UP21.log

I tried to figure out what is going on and what I saw is that the configuration for cosmic has a special digi config file and PPS is not included. What I can do it to issue another PR including PPS digi into the cosmic one, if that is ok with you all.

Thanks.

Yes, sure, thanks. Please let us know when the PR is ready.

cmsbuild · 2020-11-21T20:03:04Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

silviodonato · 2020-11-21T20:03:36Z

please test

cmsbuild · 2020-11-21T20:04:00Z

The tests are being triggered in jenkins.
Test Parameters:

MATRIX_EXTRAS = 8.2,11634.0
CMSSW_11_2_X_2020-11-21-1100/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10923/console Started: 2020/11/21 21:05

cmsbuild · 2020-11-21T21:40:36Z

+1
Tested at: 2b6266a
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-472d82/10923/summary.html
CMSSW: CMSSW_11_2_X_2020-11-21-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-11-21T21:40:38Z

Comparison job queued.

cmsbuild · 2020-11-21T23:00:14Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-472d82/10923/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-472d82/8.2_BeamHalo_UP21+BeamHalo_UP21+DIGICOS_UP21+RECOCOS_UP21+ALCABH_UP21+HARVESTCOS_UP21

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 37
DQMHistoTests: Total histograms compared: 2961011
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2960988
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 36 files compared)
Checked 158 log files, 34 edm output root files, 37 DQM output files

Martin-Grunewald · 2020-11-23T07:11:39Z

Hello,

This PR creates problems for TSG tests, eg this one:

RelVal --step=DIGI:pdigi,L1,DIGI2RAW --conditions=auto:run3_mc_GRun --filein=root://eoscms.cern.ch//eos/cms/store/group/dpg_trigger/comm_trigger/TriggerStudiesGroup/STORM/GEN-SIM/CMSSW_9_phase1/14F749AC-8AFE-E611-9821-0CC47A78A4A0.root --custom_conditions= --fileout=RelVal_DigiL1Raw_GRun_MC.root --number=100 --mc --no_exec --datatier GEN-SIM-DIGI-RAW --eventcontent=RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --era=Run3 --customise= --scenario=pp --python_filename=RelVal_DigiL1Raw_GRun_MC.py

with this error message:

----- Begin Fatal Exception 23-Nov-2020 08:07:40 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 3 event: 102 stream: 0
   [1] Running path 'RAWSIMoutput_step'
   [2] Prefetching for module PoolOutputModule/'RAWSIMoutput'
   [3] Prefetching for module RawDataCollectorByLabel/'rawDataCollector'
   [4] Prefetching for module CTPPSPixelDigiToRaw/'ctppsPixelRawData'
   [5] Calling method for module CTPPSPixelDigiProducer/'RPixDetDigitizer'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: CrossingFrame<PSimHit>
Looking for module label: mix
Looking for productInstanceName: g4SimHitsCTPPSPixelHits

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exc\
eption,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSe\
t in the configuration.

----- End Fatal Exception -------------------------------------------------

Please advise!

silviodonato · 2020-11-23T12:57:29Z

Hi @Martin-Grunewald , could you update the GEN-SIM files used in the tests to something made with CMSSW_11_2_0_pre6 or later? This should solve this problem, see #32125 #32140 #31991

mundim · 2020-11-25T18:28:28Z

Hi @Martin-Grunewald , could you update the GEN-SIM files used in the tests to something made with CMSSW_11_2_0_pre6 or later? This should solve this problem, see #32125 #32140 #31991

Hi. Is there any news? I'm worried about the integration of this PR in 11_2 release... Sorry to keep pushing this...

Thanks a lot.

Martin-Grunewald · 2020-11-26T05:20:29Z

@mundim
I have updated the GEN-SIM input files for our TSG tests as part of #32253, so all fine now.

Revert "Revert "PPS Digi integration into cmssw (final)""

2b6266a

cmsbuild added this to the CMSSW_11_2_X milestone Nov 2, 2020

cmsbuild added code-checks-pending comparison-pending operations-pending orp-pending pending-signatures tests-pending labels Nov 2, 2020

cmsbuild added the simulation-pending label Nov 2, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Nov 2, 2020

cmsbuild added tests-started and removed tests-pending labels Nov 2, 2020

silviodonato mentioned this pull request Nov 2, 2020

g4SimHitsCTPPSPixelHits not found #31991

Closed

cmsbuild added tests-rejected and removed tests-started labels Nov 2, 2020

cmsbuild mentioned this pull request Nov 4, 2020

Ecal phase2 new digitization 11 2 x #31726

Merged

cmsbuild added fully-signed orp-approved and removed orp-pending pending-signatures labels Nov 21, 2020

cmsbuild merged commit 75a3705 into master Nov 21, 2020

cmsbuild added comparison-pending tests-pending tests-started and removed comparison-available tests-approved requires-external tests-pending labels Nov 21, 2020

cmsbuild added tests-approved and removed tests-started labels Nov 21, 2020

cmsbuild added comparison-available and removed comparison-pending labels Nov 21, 2020

silviodonato mentioned this pull request Nov 30, 2020

No "LHCInfoRcd" record found in the EventSetup.n (CTPPSProtonProducer/'ctppsProtons') #32340

Closed

smuzaffar deleted the revert-31997-revert-31943-PPSDigi4Run3 branch January 13, 2021 10:24

This was referenced Mar 25, 2021

Re-enable ctppsProtons from recoCTPPSTask (Run-3 only) #33265

Closed

Re-enable ctppsProtons from recoCTPPSTask in Run-3 #33266

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Restore "PPS Digi integration into cmssw (final)" #32003

Restore "PPS Digi integration into cmssw (final)" #32003

silviodonato commented Nov 2, 2020

cmsbuild commented Nov 2, 2020

silviodonato commented Nov 2, 2020

silviodonato commented Nov 2, 2020

cmsbuild commented Nov 2, 2020

silviodonato commented Nov 2, 2020

cmsbuild commented Nov 2, 2020

cmsbuild commented Nov 2, 2020

silviodonato commented Nov 2, 2020

cmsbuild commented Nov 2, 2020 •

edited

Loading

cmsbuild commented Nov 2, 2020

cmsbuild commented Nov 2, 2020

mundim commented Nov 3, 2020

silviodonato commented Nov 6, 2020

cmsbuild commented Nov 21, 2020

silviodonato commented Nov 21, 2020

cmsbuild commented Nov 21, 2020 •

edited

Loading

cmsbuild commented Nov 21, 2020

cmsbuild commented Nov 21, 2020

cmsbuild commented Nov 21, 2020

Martin-Grunewald commented Nov 23, 2020

silviodonato commented Nov 23, 2020

mundim commented Nov 25, 2020

Martin-Grunewald commented Nov 26, 2020

Restore "PPS Digi integration into cmssw (final)" #32003

Restore "PPS Digi integration into cmssw (final)" #32003

Conversation

silviodonato commented Nov 2, 2020

PR description:

cmsbuild commented Nov 2, 2020

silviodonato commented Nov 2, 2020

silviodonato commented Nov 2, 2020

cmsbuild commented Nov 2, 2020

silviodonato commented Nov 2, 2020

cmsbuild commented Nov 2, 2020

cmsbuild commented Nov 2, 2020

silviodonato commented Nov 2, 2020

cmsbuild commented Nov 2, 2020 • edited Loading

cmsbuild commented Nov 2, 2020

cmsbuild commented Nov 2, 2020

mundim commented Nov 3, 2020

silviodonato commented Nov 6, 2020

cmsbuild commented Nov 21, 2020

silviodonato commented Nov 21, 2020

cmsbuild commented Nov 21, 2020 • edited Loading

cmsbuild commented Nov 21, 2020

cmsbuild commented Nov 21, 2020

cmsbuild commented Nov 21, 2020

Martin-Grunewald commented Nov 23, 2020

silviodonato commented Nov 23, 2020

mundim commented Nov 25, 2020

Martin-Grunewald commented Nov 26, 2020

cmsbuild commented Nov 2, 2020 •

edited

Loading

cmsbuild commented Nov 21, 2020 •

edited

Loading