Adding Run 330098 (2019 MWGR3) in data relvals #27341

boudoul · 2019-06-25T15:45:24Z

PR description:

Greetings
This PR aims to introduce a new run 3 data workflow based on fresh run3 data taken during MWGR3 .
The goal is to test the run3 reconstruction with real data and anticipate any issue without waiting for the usual Tier0 replay prior the global runs - Eventually we can update with more fresh runs once we get some.

PR validation:

I ran the following workflow runTheMatrix.py -l 138.1 --command "-n 100"

if this PR is a backport please specify the original PR:

It may need to be backported in 10_6 (depending on the MWGR 4 plan) but let's make a review in master first

boudoul · 2019-06-25T15:45:33Z

I would need :

comments from PdmV (@pgunnell,@prebello,@zhenhu) : if they are happy with the way I'm introducing this WF ( WF number, naming etc..)
comments from Alca ( @christopheralanwest,@franzoni,@tlampen,@pohsun,@tocheng ) : I used a candidate GT kindly prepared by @christopheralanwest a couple of weeks ago, and defined it as run3_data in autocond. Would it be possible to get a 'real GT' ( so that I can update this PR with it and not leave a candidate GT in this workflow ) - Thanks.

cmsbuild · 2019-06-25T15:45:56Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-06-25T15:47:43Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-06-25T15:54:11Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27341/10598

This PR adds an extra 52KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/AlCa/python/autoCond.py modified in PR(s): Update 2021 global tags to include new pixel conditions #27334
- File Configuration/PyReleaseValidation/python/relval_standard.py modified in PR(s): Add bx skim and corresponding RelVal #26476
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Add bx skim and corresponding RelVal #26476, Override the default value of the input file for wf 250203.18 #27336

cmsbuild · 2019-06-25T15:54:40Z

A new Pull Request was created by @boudoul (boudoul) for master.

It involves the following packages:

Configuration/AlCa
Configuration/PyReleaseValidation

@cmsbuild, @prebello, @zhenhu, @christopheralanwest, @tocheng, @pgunnell, @franzoni, @kpedro88, @tlampen, @pohsun can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @mmusich, @tocheng this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

mmusich · 2019-06-25T16:33:14Z

Configuration/AlCa/python/autoCond.py

@@ -44,6 +44,8 @@
    # GlobalTag for Run2 HLT for HI (not 2018 HI): it points to the online GT
    'run2_hlt_hi'       :   '101X_dataRun2_HLTHI_frozen_v9',
    # GlobalTag for MC production with perfectly aligned and calibrated detector for Phase1 2017 (and 0,0,~0-centred beamspot)
+    'run3_data'         :   '106X_dataRun3_Express_Candidate_2019_05_24_21_36_45',


@boudoul in the past 'run2_data' key was reserved for offline / re-reco Global Tags. I think to preserve the nomenclature something like run3_data_promptlike would be more appropriate. By the way do we have a Run3 prompt queue?

I agree that run3_data should be reserved for offline global tags.

@boudoul Do we plan to have prompt reco for the MWGRs or only express? We do have both express and prompt queues for Run 3: "106X_dataRun3_Express_Queue" and "106X_dataRun3_Prompt_Queue". But if we are not planning to have prompt reco in the near future, we should have an express GT here: there's no point in validating a GT that we don't plan to use. I would be inclined to use the key name run3_data_promptlike in either case.

fabiocos · 2019-06-25T17:00:32Z

The overlap with #27336 is not going to produce conflicts

christopheralanwest · 2019-06-25T21:36:17Z

Configuration/AlCa/python/autoCond.py

@@ -44,6 +44,8 @@
    # GlobalTag for Run2 HLT for HI (not 2018 HI): it points to the online GT
    'run2_hlt_hi'       :   '101X_dataRun2_HLTHI_frozen_v9',
    # GlobalTag for MC production with perfectly aligned and calibrated detector for Phase1 2017 (and 0,0,~0-centred beamspot)
+    'run3_data'         :   '106X_dataRun3_Express_Candidate_2019_05_24_21_36_45',


I agree that run3_data should be reserved for offline global tags.

@boudoul Do we plan to have prompt reco for the MWGRs or only express? We do have both express and prompt queues for Run 3: "106X_dataRun3_Express_Queue" and "106X_dataRun3_Prompt_Queue". But if we are not planning to have prompt reco in the near future, we should have an express GT here: there's no point in validating a GT that we don't plan to use. I would be inclined to use the key name run3_data_promptlike in either case.

christopheralanwest · 2019-06-25T21:47:08Z

Configuration/AlCa/python/autoCond.py

@@ -44,6 +44,8 @@
    # GlobalTag for Run2 HLT for HI (not 2018 HI): it points to the online GT
    'run2_hlt_hi'       :   '101X_dataRun2_HLTHI_frozen_v9',
    # GlobalTag for MC production with perfectly aligned and calibrated detector for Phase1 2017 (and 0,0,~0-centred beamspot)
+    'run3_data'         :   '106X_dataRun3_Express_Candidate_2019_05_24_21_36_45',
+    # GlobalTag for Run3 data relvals


The comments in L46 and L48 should be switched so that the comment is above the corresponding key-value pair. Please try to maintain the same whitespace in the unchanged lines to avoid irrelevant merge conflicts.

Hi @christopheralanwest , thanks for your comments (sorry for the delay, CMS week..) We should consider having prompt reconstruction in MWGR, do you have a name of a GT I should use ? (I will also use un3_data_promptlike as you recommended) - And for the line switching , thanks for spotting , it was not intentionnal to change , I will fix in a next commit. Thanks again .

boudoul · 2019-07-02T08:04:16Z

Dear Alca ( @christopheralanwest,@franzoni,@tlampen,@pohsun,@tocheng )- Could you please let me know the name of the GT I should use, in order for me to edit this PR with this new GT and also take care of the comments at the same time. MWGR is approaching ....
Thank you .

tlampen · 2019-07-02T08:29:21Z

Dear Gaëlle, my understanding is that it should be 106X_upgrade2021_realistic_v8 . I hope @christopheralanwest can confirm to be sure.

boudoul · 2019-07-02T08:32:55Z

Hi @tlampen , thanks. This sounds more like a MC GT and not a data GT, no ? Anyway I let @christopheralanwest comment when he can in these busy days .

christopheralanwest · 2019-07-02T13:43:50Z

We currently don't have any better data GT for MWGR#1 than the candidate 106X_dataRun3_Express_Candidate_2019_05_24_21_36_45 that you have proposed. When MWGR#1 was taken, proper Run-3 HCAL conditions for data were not yet available. The candidate GT referenced here includes HCAL test tags with IOV from [1, inf) so that a test of the new HCAL conditions could be performed. But because of synchronization rules needed to ensure reproducibility, we cannot append new IOVs to a tag with an express synchronization with an IOV later than (last run taken at P5+1).

cmsbuild · 2019-07-26T19:02:18Z

Comparison job queued.

cmsbuild · 2019-07-26T21:03:29Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a1ff85/1659/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-a1ff85/138.1_RunCosmics2019+RunCosmics2019+RECOCOSDRUN3+ALCACOSDRUN3+HARVESTDCRUN3

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 32
DQMHistoTests: Total histograms compared: 2628546
DQMHistoTests: Total failures: 2
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2628227
DQMHistoTests: Total skipped: 317
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 31 files compared)
Checked 133 log files, 14 edm output root files, 32 DQM output files

christopheralanwest · 2019-07-27T12:30:14Z

+1

boudoul · 2019-07-29T07:02:12Z

Thanks @christopheralanwest !
@fabiocos , can we unhold now ? and if the remaining signatures (@prebello, @zhenhu, @pgunnell, @kpedro88) could also take a look , that's would be great. Thanks.

kpedro88 · 2019-07-29T13:37:58Z

+upgrade

kpedro88 · 2019-07-29T13:39:12Z

unhold

boudoul · 2019-07-30T06:11:48Z

Hi PdmV (@prebello, @zhenhu, @pgunnell ), we need your comments here and eventually your signature

prebello · 2019-07-30T06:37:26Z

+1

all looks ok. 138.1 tested and approved. sorry for the delay @boudoul. This github announcement skipped my daily check.

cmsbuild · 2019-07-30T06:37:54Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

boudoul · 2019-07-30T06:39:28Z

Thanks @prebello !

kpedro88 · 2019-07-30T21:47:42Z

+1

cmsbuild added this to the CMSSW_11_0_X milestone Jun 25, 2019

cmsbuild added alca-pending code-checks-pending comparison-pending orp-pending pdmv-pending pending-signatures tests-pending upgrade-pending labels Jun 25, 2019

cmsbuild mentioned this pull request Jun 25, 2019

Update 2021 global tags to include new pixel conditions #27334

Merged

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 25, 2019

mmusich reviewed Jun 25, 2019

View reviewed changes

christopheralanwest reviewed Jun 25, 2019

View reviewed changes

cmsbuild added code-checks-pending and removed code-checks-approved labels Jul 2, 2019

cmsbuild added tests-approved and removed tests-started labels Jul 26, 2019

cmsbuild added comparison-available and removed comparison-pending labels Jul 26, 2019

cmsbuild added alca-approved and removed alca-pending labels Jul 27, 2019

cmsbuild added upgrade-approved and removed upgrade-pending labels Jul 29, 2019

cmsbuild removed the hold label Jul 29, 2019

cmsbuild added fully-signed pdmv-approved and removed pdmv-pending pending-signatures labels Jul 30, 2019

cmsbuild added orp-approved and removed orp-pending labels Jul 30, 2019

cmsbuild merged commit 38ee57b into cms-sw:master Jul 30, 2019

cmsbuild mentioned this pull request Jul 31, 2019

DD: Replace PseudoTrap shape by TruncTubs #27632

Merged

mmusich mentioned this pull request Mar 10, 2020

Strips Offline code: TCDS-Scal migration #29116

Merged

christopheralanwest mentioned this pull request Aug 2, 2020

Introduce Run 3 cosmics workflow using express GT #31020

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding Run 330098 (2019 MWGR3) in data relvals #27341

Adding Run 330098 (2019 MWGR3) in data relvals #27341

boudoul commented Jun 25, 2019 •

edited

boudoul commented Jun 25, 2019

cmsbuild commented Jun 25, 2019

cmsbuild commented Jun 25, 2019

cmsbuild commented Jun 25, 2019

cmsbuild commented Jun 25, 2019

mmusich Jun 25, 2019

christopheralanwest Jun 25, 2019

fabiocos commented Jun 25, 2019

christopheralanwest Jun 25, 2019

christopheralanwest Jun 25, 2019

boudoul Jun 27, 2019

boudoul commented Jul 2, 2019

tlampen commented Jul 2, 2019

boudoul commented Jul 2, 2019

christopheralanwest commented Jul 2, 2019

cmsbuild commented Jul 26, 2019

cmsbuild commented Jul 26, 2019

christopheralanwest commented Jul 27, 2019

boudoul commented Jul 29, 2019

kpedro88 commented Jul 29, 2019

kpedro88 commented Jul 29, 2019

boudoul commented Jul 30, 2019

prebello commented Jul 30, 2019

cmsbuild commented Jul 30, 2019

boudoul commented Jul 30, 2019

kpedro88 commented Jul 30, 2019

Adding Run 330098 (2019 MWGR3) in data relvals #27341

Adding Run 330098 (2019 MWGR3) in data relvals #27341

Conversation

boudoul commented Jun 25, 2019 • edited

PR description:

PR validation:

if this PR is a backport please specify the original PR:

boudoul commented Jun 25, 2019

cmsbuild commented Jun 25, 2019

cmsbuild commented Jun 25, 2019

cmsbuild commented Jun 25, 2019

cmsbuild commented Jun 25, 2019

mmusich Jun 25, 2019

Choose a reason for hiding this comment

christopheralanwest Jun 25, 2019

Choose a reason for hiding this comment

fabiocos commented Jun 25, 2019

christopheralanwest Jun 25, 2019

Choose a reason for hiding this comment

christopheralanwest Jun 25, 2019

Choose a reason for hiding this comment

boudoul Jun 27, 2019

Choose a reason for hiding this comment

boudoul commented Jul 2, 2019

tlampen commented Jul 2, 2019

boudoul commented Jul 2, 2019

christopheralanwest commented Jul 2, 2019

cmsbuild commented Jul 26, 2019

cmsbuild commented Jul 26, 2019

christopheralanwest commented Jul 27, 2019

boudoul commented Jul 29, 2019

kpedro88 commented Jul 29, 2019

kpedro88 commented Jul 29, 2019

boudoul commented Jul 30, 2019

prebello commented Jul 30, 2019

cmsbuild commented Jul 30, 2019

boudoul commented Jul 30, 2019

kpedro88 commented Jul 30, 2019

boudoul commented Jun 25, 2019 •

edited