Adding GEN fragment (GluGluTo2Jets_ExHume) specific for PPS wf testing #32765

mundim · 2021-01-28T17:35:32Z

PR description:

Add a GEN fragment (ExHume generator, gluongluon -> 2 Jets, 14 TeV, central mass between 300 and 2000 GeV) to produce protons inside PPS acceptance to allow it to be properly tested.

PR validation:

scram b runtests
scram b code-checks
scram b code-format

ran without any issues.

standard runTheMatrix --jobsReports -l limited --ibeos
produced no error

runTheMatrix.py --what upgrade -l 11725.0,11925.0,11925.98,11925.99,12125.0,12325.0,12325.98,12325.99 -t 4 --command='-n 5

where the wf are the ones in which GluGluTo2Jets is present (as advised in #32352), produced no error for the ones with NO PU, but it was reported missing files for the PU ones

cmsbuild · 2021-01-28T17:42:48Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32765/20923

This PR adds an extra 36KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Track classifier using TensorFlow - continuation #32128, Phase2-gex48 Add 2 new scenarios D75 and D76 using I13 for D71 and D74. Also remove some overlaps #32738, Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, Patatrack integration - Pixel workflows (12/N) #31854, Introduce TritonService and workflows #32576

cmsbuild · 2021-01-28T17:43:11Z

A new Pull Request was created by @mundim for master.

It involves the following packages:

Configuration/Generator
Configuration/PyReleaseValidation

@SiewYan, @mkirsano, @jordan-martins, @chayanit, @wajidalikhan, @kpedro88, @cmsbuild, @GurpreetSinghChahal, @srimanob, @agrohsje, @alberto-sanchez can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @makortel, @fabiocos, @slomeo this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

srimanob · 2021-01-28T18:10:38Z

test parameters:

workflows = 11725.0,11925.0,11925.98,11925.99,12125.0,12325.0,12325.98,12325.99

srimanob · 2021-01-28T18:11:34Z

Please test

mundim · 2021-01-28T18:43:51Z

the wf 12325.0, 11925.0, 12325.98, 11925.98, 12325.99 and 11925.99 (the ones with PU) failed in my area because of a missing secondary file. I hope it will not happen in the official testing area.

cmsbuild · 2021-01-28T19:18:44Z

-1

Failed Tests: RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-dfda86/12591/summary.html
COMMIT: 4425c67
CMSSW: CMSSW_11_3_X_2021-01-28-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/32765/12591/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

mundim · 2021-01-28T20:12:33Z

@chayanit sorry, I forgot to mention you here.

chayanit · 2021-01-29T10:46:39Z

Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py

@@ -1288,4 +1288,5 @@ def __init__(self, howMuch, dataset):
    ('TenTau_E_15_500_Eta3p1_pythia8_cfi', UpgradeFragment(Kby(9,100),'TenTau_15_500_Eta3p1')),
    ('QCD_Pt_1800_2400_14TeV_TuneCP5_cfi', UpgradeFragment(Kby(9,50), 'QCD_Pt_1800_2400_14')),
    ('DisplacedSUSY_stopToBottom_M_800_500mm_TuneCP5_14TeV_pythia8_cff', UpgradeFragment(Kby(9,50),'DisplacedSUSY_stopToB_M_800_500mm_14')),


Would you mind to help me also update this line so I don't need to open new PR?
('DisplacedSUSY_stopToBottom_M_800_500mm_TuneCP5_14TeV_pythia8_cff', UpgradeFragment(Kby(9,50),'DisplacedSUSY_14TeV')),

chayanit · 2021-01-29T10:47:35Z

Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py

@@ -1288,4 +1288,5 @@ def __init__(self, howMuch, dataset):
    ('TenTau_E_15_500_Eta3p1_pythia8_cfi', UpgradeFragment(Kby(9,100),'TenTau_15_500_Eta3p1')),
    ('QCD_Pt_1800_2400_14TeV_TuneCP5_cfi', UpgradeFragment(Kby(9,50), 'QCD_Pt_1800_2400_14')),
    ('DisplacedSUSY_stopToBottom_M_800_500mm_TuneCP5_14TeV_pythia8_cff', UpgradeFragment(Kby(9,50),'DisplacedSUSY_stopToB_M_800_500mm_14')),
+    ('GluGluTo2Jets_M_300_2000_14TeV_Exhume_cff',UpgradeFragment(Kby(9,100),'GluGluTo2Jets_M_300_2000_14TeV')),


I'm afraid the string is a bit too long. I would request to change to ('GluGluTo2Jets_M_300_2000_14TeV_Exhume_cff',UpgradeFragment(Kby(9,100),'GluGluTo2Jets_14TeV')),

chayanit · 2021-01-29T10:49:59Z

please test
@srimanob I don't think we can ask bot to test new workflows.

chayanit · 2021-01-29T10:50:41Z

@chayanit sorry, I forgot to mention you here.

no need. I'm included as L2 PdmV to assign this PR already. Could you commit changes as I requested? Thanks.

chayanit · 2021-02-17T15:43:17Z

Hi @chayanit. I was trying to understand this... I'm not sure about your question. The wf ID I took from runTheMatrix.py --what upgrade --showMatrix | grep -i GluGluTo2Jets_M | grep 2021. But it seems the PU wf is trying to get a MinBias from 2011, in which certainly, PPS was not present and I imagine a mixing will fail, right?

@mundim Can you show me what you got after run runTheMatrix.py --what upgrade --showMatrix | grep -i GluGluTo2Jets_M | grep 2021 only for these 4 workflows 11725.0, 11925.0 and 12125.0, 12325.0?

mundim · 2021-02-17T17:40:23Z

Hi. this is the output:

11725.0 2021+GluGluTo2Jets_M_300_2000_14TeV_Exhume_GenSim+Digi+Reco+HARVEST+ALCA
11925.0 2021PU+GluGluTo2Jets_M_300_2000_14TeV_Exhume_GenSim+DigiPU+RecoPU+HARVESTPU+Nano
12125.0 2021Design+GluGluTo2Jets_M_300_2000_14TeV_Exhume_GenSim+Digi+Reco+HARVEST
12325.0 2021DesignPU+GluGluTo2Jets_M_300_2000_14TeV_Exhume_GenSim+DigiPU+RecoPU+HARVESTPU

chayanit · 2021-02-18T00:56:53Z

Hi. this is the output:

11725.0 2021+GluGluTo2Jets_M_300_2000_14TeV_Exhume_GenSim+Digi+Reco+HARVEST+ALCA
11925.0 2021PU+GluGluTo2Jets_M_300_2000_14TeV_Exhume_GenSim+DigiPU+RecoPU+HARVESTPU+Nano
12125.0 2021Design+GluGluTo2Jets_M_300_2000_14TeV_Exhume_GenSim+Digi+Reco+HARVEST
12325.0 2021DesignPU+GluGluTo2Jets_M_300_2000_14TeV_Exhume_GenSim+DigiPU+RecoPU+HARVESTPU

OK. We should select 11725.0 (noPU) and 11925.0 (PU) as they are standard scenario and not the Design ones. Can you update in relval_2017 file to these two workflows instead? @mundim

Configuration/PyReleaseValidation/python/relval_2017.py

mundim · 2021-02-18T14:40:07Z

I solved the conflict about relval_2017, but I'm not sure about the other 2 files (I used the latest IB as the base).

cmsbuild · 2021-02-18T14:40:40Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32765/21176

This PR adds an extra 40KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_2017.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, Patatrack integration - Pixel workflows (12/N) #31854
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, Patatrack integration - Pixel workflows (12/N) #31854
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, PPS pixel topology #32868, Patatrack integration - Pixel workflows (12/N) #31854

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-checks:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32765/21176/code-checks.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32765/21176/code-checks.patch | patch -p1
You can also run scram build code-checks to apply code checks directly

chayanit · 2021-02-23T06:47:13Z

Hello @mundim , @qliphy , @silviodonato any problem with this PR? It seems to conflict with others. How should we fix and move this PR?

silviodonato · 2021-02-23T10:02:48Z

After having merged #32969 and #32868 everything should be already merged in master.
@mundim could you check all the code is included in https://github.com/cms-sw/cmssw ?

mundim · 2021-02-23T20:07:27Z

After having merged #32969 and #32868 everything should be already merged in master.
@mundim could you check all the code is included in https://github.com/cms-sw/cmssw ?

Some how the proposed change in realval_2017.py have been ignored as well as some corrections in the SM parameters in the generator fragment.

qliphy · 2021-02-24T02:02:13Z

After having merged #32969 and #32868 everything should be already merged in master.
@mundim could you check all the code is included in https://github.com/cms-sw/cmssw ?

Some how the proposed change in realval_2017.py have been ignored as well as some corrections in the SM parameters in the generator fragment.

@mundim As mentioned in #32969 (comment) you can open a new PR to add all missing fixes. Thanks!

silviodonato · 2021-02-24T11:56:57Z

@mundim
I squashed the commits of this PR and fixed the conflict with master.
Does https://github.com/cms-sw/cmssw/compare/master...silviodonato:PPSRelValGenFragment_reb?expand=1 look ok ?

mundim · 2021-02-24T13:04:03Z

@mundim
I squashed the commits of this PR and fixed the conflict with master.
Does https://github.com/cms-sw/cmssw/compare/master...silviodonato:PPSRelValGenFragment_reb?expand=1 look ok ?

It is ok with the gen fragment, but in the relval_2017.py file, lines 33 and 60 got added instead of changed, they are not duplicated.

silviodonato · 2021-02-24T13:17:04Z

the

@mundim
I squashed the commits of this PR and fixed the conflict with master.
Does https://github.com/cms-sw/cmssw/compare/master...silviodonato:PPSRelValGenFragment_reb?expand=1 look ok ?

It is ok with the gen fragment, but in the relval_2017.py file, lines 33 and 60 got added instead of changed, they are not duplicated.

@mundim Right, thanks a lot. I've fixed it. Will you open the PR or shall I do it?

mundim · 2021-02-24T13:35:56Z

the

@mundim
I squashed the commits of this PR and fixed the conflict with master.
Does https://github.com/cms-sw/cmssw/compare/master...silviodonato:PPSRelValGenFragment_reb?expand=1 look ok ?

It is ok with the gen fragment, but in the relval_2017.py file, lines 33 and 60 got added instead of changed, they are not duplicated.

@mundim Right, thanks a lot. I've fixed it. Will you open the PR or shall I do it?

Done already. thank you very much.

silviodonato · 2021-02-24T13:55:47Z

moved to #32983

Adding GEN fragment (GluGluTo2Jets_ExHume) specific for PPS wf testing

4425c67

cmsbuild added this to the CMSSW_11_3_X milestone Jan 28, 2021

cmsbuild added code-checks-pending generators-pending orp-pending pdmv-pending pending-signatures tests-pending upgrade-pending labels Jan 28, 2021

mundim mentioned this pull request Jan 28, 2021

re-enable ctppsProtons for Run3 after #32352 #32356

Closed

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 28, 2021

cmsbuild added tests-started and removed tests-pending labels Jan 28, 2021

cmsbuild mentioned this pull request Jan 28, 2021

Track classifier using TensorFlow - continuation #32128

Merged

cmsbuild added tests-rejected and removed tests-started labels Jan 28, 2021

cmsbuild mentioned this pull request Jan 29, 2021

Introduce TritonService and workflows #32576

Merged

chayanit suggested changes Jan 29, 2021

View reviewed changes

cmsbuild added tests-started and removed tests-rejected labels Jan 29, 2021

chayanit suggested changes Feb 18, 2021

View reviewed changes

Configuration/PyReleaseValidation/python/relval_2017.py Outdated Show resolved Hide resolved

Changing the wf number as requested in the PR 32765

66dad21

cmsbuild added code-checks-pending tests-pending and removed code-checks-approved tests-rejected labels Feb 18, 2021

cmsbuild added code-checks-rejected and removed code-checks-pending labels Feb 18, 2021

This was referenced Feb 19, 2021

Add new D78,D79 geometries based on D76 with T22,T23 #32952

Merged

Add customisation to make sure that upgrade WFs will cross LS in the short matrix #32957

Closed

[RFC] Outline of HGCAL ML ntuples using NanoAOD #32187

Closed

silviodonato mentioned this pull request Feb 23, 2021

Add missing GluGluTo2Jets_M_300_2000_14TeV_Exhume_cff.py #32969

Merged

mundim mentioned this pull request Feb 24, 2021

Adding GEN fragment (GluGluTo2Jets_ExHume) specific for PPS wf testing #32983

Merged

silviodonato closed this Feb 24, 2021

This was referenced Mar 25, 2021

Re-enable ctppsProtons from recoCTPPSTask (Run-3 only) #33265

Closed

Re-enable ctppsProtons from recoCTPPSTask in Run-3 #33266

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding GEN fragment (GluGluTo2Jets_ExHume) specific for PPS wf testing #32765

Adding GEN fragment (GluGluTo2Jets_ExHume) specific for PPS wf testing #32765

mundim commented Jan 28, 2021

cmsbuild commented Jan 28, 2021

cmsbuild commented Jan 28, 2021

srimanob commented Jan 28, 2021

srimanob commented Jan 28, 2021

mundim commented Jan 28, 2021

cmsbuild commented Jan 28, 2021

mundim commented Jan 28, 2021

chayanit Jan 29, 2021

mundim Jan 29, 2021

chayanit Jan 29, 2021

chayanit commented Jan 29, 2021

chayanit commented Jan 29, 2021

chayanit commented Feb 17, 2021 •

edited

mundim commented Feb 17, 2021 •

edited

chayanit commented Feb 18, 2021

mundim commented Feb 18, 2021

cmsbuild commented Feb 18, 2021

chayanit commented Feb 23, 2021

silviodonato commented Feb 23, 2021

mundim commented Feb 23, 2021

qliphy commented Feb 24, 2021

silviodonato commented Feb 24, 2021

mundim commented Feb 24, 2021

silviodonato commented Feb 24, 2021

mundim commented Feb 24, 2021

silviodonato commented Feb 24, 2021

Adding GEN fragment (GluGluTo2Jets_ExHume) specific for PPS wf testing #32765

Adding GEN fragment (GluGluTo2Jets_ExHume) specific for PPS wf testing #32765

Conversation

mundim commented Jan 28, 2021

PR description:

PR validation:

cmsbuild commented Jan 28, 2021

cmsbuild commented Jan 28, 2021

srimanob commented Jan 28, 2021

srimanob commented Jan 28, 2021

mundim commented Jan 28, 2021

cmsbuild commented Jan 28, 2021

RelVals

mundim commented Jan 28, 2021

chayanit Jan 29, 2021

Choose a reason for hiding this comment

mundim Jan 29, 2021

Choose a reason for hiding this comment

chayanit Jan 29, 2021

Choose a reason for hiding this comment

chayanit commented Jan 29, 2021

chayanit commented Jan 29, 2021

chayanit commented Feb 17, 2021 • edited

mundim commented Feb 17, 2021 • edited

chayanit commented Feb 18, 2021

mundim commented Feb 18, 2021

cmsbuild commented Feb 18, 2021

chayanit commented Feb 23, 2021

silviodonato commented Feb 23, 2021

mundim commented Feb 23, 2021

qliphy commented Feb 24, 2021

silviodonato commented Feb 24, 2021

mundim commented Feb 24, 2021

silviodonato commented Feb 24, 2021

mundim commented Feb 24, 2021

silviodonato commented Feb 24, 2021

chayanit commented Feb 17, 2021 •

edited

mundim commented Feb 17, 2021 •

edited