Update L1 menu in HLT relval and Prompt Like GTs #25068

tocheng · 2018-10-31T07:46:01Z

The PR is to update GTs according to the L1 menu change in HLT relval and prompt like autoConds in PR #25029 and #25030.

Two keys, run2_hlt_relval_hi and run2_data_promptlike_hi are added for HLT relval and test reco GTs for heavy Ion data taking.

The update in HLT relval GT can be seen at
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/103X_dataRun2_HLT_relval_HI_v1/103X_dataRun2_HLT_relval_v4
Besides L1 menu, the differences are CTPPS local reconstruction, GEM emap and geometry, JEC, HBHE negative energy filter and using HLT-synchronized track MVA selector.

The update in Prompt like GT can be seen at
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/103X_dataRun2_PromptLike_HI_v1/103X_dataRun2_PromptLike_v6
Besides L1 menu, the main updates includes GEM emap and RPC CPPF link.

The updated in MC (upgrade2018_realistic) can be seen at
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/103X_upgrade2018_realistic_HI_v9/103X_upgrade2018_realistic_HI_v8

The PR doesn't update the HLT relval and Prompt-like GT.

cmsbuild · 2018-10-31T07:46:36Z

The code-checks are being triggered in jenkins.

cmsbuild · 2018-10-31T07:49:18Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-25068/7062

cmsbuild · 2018-10-31T07:49:34Z

A new Pull Request was created by @tocheng (Tongguang) for master.

It involves the following packages:

Configuration/AlCa

@cmsbuild, @franzoni, @pohsun, @tocheng, @lpernie can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @mmusich, @tocheng this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

tocheng · 2018-10-31T07:50:04Z

@cmsbuild please test

cmsbuild · 2018-10-31T07:50:18Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/31392/console Started: 2018/10/31 08:50

Martin-Grunewald · 2018-10-31T08:30:34Z

Sorry, but this does not look correct to me:

You replaced the 2018 pp L1T menu L1Menu_Collisions2018_v2_1_0-d1_xml by the L1T HIon menu,
but this should be done ONLY in HIon-specific GTs, not pp GTs, like for MC where we have phase1_2018_realistic for pp and phase1_2018_realistic_hi for HI - and thus also phase1_2018_realistic_hi should get the new HIon L1T menu!

Could you please elaborate?

??

tocheng · 2018-10-31T08:34:21Z

Sorry, but this does not look correct to me:

You replaced the 2018 pp L1T menu L1Menu_Collisions2018_v2_1_0-d1_xml by the L1T HIon menu,
but this should be done ONLY in HIon-specific GTs, not pp GTs, like for MC where we have phase1_2018_realistic for pp and phase1_2018_realistic_hi for HI - and thus also phase1_2018_realistic_hi should get the new HIon L1T menu!

Could you please elaborate?

??

Hello @Martin-Grunewald ,

I see in PR #25030 ,
the heavyIon L1 menu is associated to 'run2_hlt_relval' and 'run2_data_promptlike' in the autoHLT
run2_hlt_HIon' : ('run2_hlt_relval' ,l1Menus['HIon'])
'run2_data_HIon' : ('run2_data_promptlike' ,l1Menus['HIon'])

So I updated the GTs in the autoCond for 'run2_hlt_relval' and 'run2_data_promptlike' in the PR.

Martin-Grunewald · 2018-10-31T08:38:46Z

Yes, that was my kludge to get it to work for the internal TSG tests (these GTs are NOT used outside TSG), but as expressed in the e-mail thread, it is not really correct - see especially the last paragraph quoted below!

e-mail thread: 2018 HIon menu in GTs and RelVals

PRs 25029 (104X) and 25030 (103X) replace the old (placeholder)
HLT HIon menu by the 2018 version. This HLT menu runs off the
L1T menu L1Menu_CollisionsHeavyIons2018_v4_0_0-d1_xml.

As the jenkins tests pass, I can only assume that (2018) RelVal
HIon workflows use the HLT:Fake2 menu (as the L1T menu is not
part of any GT yet and HLT HIon would crash without it), rather
than HLT:HIon.

I would assume that once the PRs are merged, the following steps
should be taken:

1) add the L1Menu_CollisionsHeavyIons2018_v4_0_0-d1_xml to the relevant GTs
2) Update the 2018 HIon RelVal tests to use HLT:HIon
3) Update all old (eg 2015/6) HIon Relvals to use HLT:Fake2

It is not clear to me which are the relevant GTs which should be affected;
in the TSG tests run offline I use these GTs for 2018 HIon tests:
  MC GT: phase1_2018_realistic_hi
  Real data HLT GT: run2_hlt_relval
  Real data Reco GT: run2_data_promptlike
(See also file  Configuration/HLT/python/autoCondHLT.py in the PR:
https://github.com/cms-sw/cmssw/pull/25029/files#diff-0e32bec8fef95e9e867873
2f08ad0e59 )

However, the latter two GTs (the real data ones) are those from
pp so one can not really replace the L1T menu in them. Are there
others to use? Even though the 2018 HIon uses --scenario=pp and
--era=Run2_2018_pp_on_AA, one may still have to make and use 
separate GTs (and GT queues).

Martin-Grunewald · 2018-10-31T08:44:27Z

maybe make a clone of run2_hlt_relval and run2_data_promptlike, call them run2_hlt_relval _hi and run2_data_promptlike_hi, with the only difference so far the L1T menu???
Similar to MC phase1_2018_realistic vs phase1_2018_realistic_hi?

??

tocheng · 2018-10-31T08:46:48Z

Yes, that was my kludge to get it to work for the internal TSG tests (these GTs are NOT used outside TSG), but as expressed in the e-mail thread, it is not really correct:

e-mail thread: 2018 HIon menu in GTs and RelVals

PRs 25029 (104X) and 25030 (103X) replace the old (placeholder)
HLT HIon menu by the 2018 version. This HLT menu runs off the
L1T menu L1Menu_CollisionsHeavyIons2018_v4_0_0-d1_xml.

As the jenkins tests pass, I can only assume that (2018) RelVal
HIon workflows use the HLT:Fake2 menu (as the L1T menu is not
part of any GT yet and HLT HIon would crash without it), rather
than HLT:HIon.

I would assume that once the PRs are merged, the following steps
should be taken:

1) add the L1Menu_CollisionsHeavyIons2018_v4_0_0-d1_xml to the relevant GTs
2) Update the 2018 HIon RelVal tests to use HLT:HIon
3) Update all old (eg 2015/6) HIon Relvals to use HLT:Fake2

It is not clear to me which are the relevant GTs which should be affected;
in the TSG tests run offline I use these GTs for 2018 HIon tests:
  MC GT: phase1_2018_realistic_hi
  Real data HLT GT: run2_hlt_relval
  Real data Reco GT: run2_data_promptlike
(See also file  Configuration/HLT/python/autoCondHLT.py in the PR:
https://github.com/cms-sw/cmssw/pull/25029/files#diff-0e32bec8fef95e9e867873
2f08ad0e59 )

However, the latter two GTs (the real data ones) are those from
pp so one can not really replace the L1T menu in them. Are there
others to use? Even though the 2018 HIon uses --scenario=pp and
--era=Run2_2018_pp_on_AA, one may still have to make and use 
separate GTs (and GT queues).

@Martin-Grunewald
We don't have HI HLT relval or HI Prompt like keys in the autoCond.
I can add them and move the GTs containing the HI L1 menu to the HI HLT relval and HI Prompt like.
Does it mean you also need to update the 2018 HI menu PR in 10_3 and 10_4 correspondingly?

Martin-Grunewald · 2018-10-31T08:51:49Z

Yes, sure, I would follow your lead to have it corrected in our TSG setup.
But please also update the already existing MC GT phase1_2018_realistic_hi with the new L1T menu.

tocheng · 2018-10-31T08:52:39Z

Yes, sure, I would follow your lead to have it corrected in our TSG setup.
But please also update the already existing MC GT phase1_2018_realistic_hi with the new L1T menu.

Yes I will. Sorry for the confusion.

tocheng · 2018-10-31T08:53:00Z

@cmsbuild please abort

cmsbuild · 2018-10-31T08:53:17Z

Jenkins tests are aborted.

…_relval and prompt-like GTs, update the HI L1 menu for HI 2018 upgrade realistic GT

tocheng · 2018-10-31T09:17:20Z

@Martin-Grunewald
Could you please double check the update is fine?
Thanks.

cmsbuild · 2018-10-31T09:19:30Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-25068/7063

cmsbuild · 2018-10-31T09:19:48Z

Pull request #25068 was updated. @cmsbuild, @franzoni, @pohsun, @tocheng, @lpernie can you please check and sign again.

Martin-Grunewald · 2018-10-31T09:24:52Z

Looks good to me, thanks!

tocheng · 2018-10-31T09:26:44Z

@cmsbuild please test

cmsbuild · 2018-10-31T09:27:03Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/31395/console Started: 2018/10/31 10:27

cmsbuild · 2018-10-31T11:44:35Z

+1
Tested at: 4db9cf2
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-25068/31395/summary.html

cmsbuild · 2018-10-31T11:44:39Z

Comparison job queued.

cmsbuild · 2018-10-31T12:56:42Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-25068/31395/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 32
DQMHistoTests: Total histograms compared: 2994843
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2994645
DQMHistoTests: Total skipped: 197
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -1.148 KiB( 31 files compared)
DQMHistoSizes: changed ( 158.0 ): -1.148 KiB L1T/L1TStage2uGT
Checked 134 log files, 14 edm output root files, 32 DQM output files

fabiocos · 2018-10-31T15:10:16Z

If I move the 2018 HI HLT menu to HIon in relval_steps, I see that both wf 300.0 and 158.0 complete successfully and finds the L1 menu. A number of warnings need to be checked, but the basic functionality looks to be there. As soon as this PR is ready I will update the relval_steps @prebello @zhenhu

tocheng · 2018-10-31T15:52:18Z

+1

cmsbuild · 2018-10-31T15:53:41Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

fabiocos · 2018-10-31T16:57:48Z

@tocheng unless you have further updates, I think we could consider this PR to be merged and move forward with the remaining updates

tocheng · 2018-10-31T17:02:57Z

@tocheng unless you have further updates, I think we could consider this PR to be merged and move forward with the remaining updates

Hello @fabiocos , I don't have further updates.
On Tuesday, we mentioned the PR to update offline data GT.
But I prefer to have a separate PR for it in order not to delay the next CMSSW_10_3_X release

fabiocos · 2018-10-31T17:48:43Z

+1

update L1 menu in HLT relval and Prompt Like GTs

9639101

cmsbuild added this to the CMSSW_10_4_X milestone Oct 31, 2018

cmsbuild added alca-pending code-checks-pending comparison-pending orp-pending pending-signatures tests-pending labels Oct 31, 2018

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 31, 2018

cmsbuild added tests-started and removed tests-pending labels Oct 31, 2018

tocheng mentioned this pull request Oct 31, 2018

Update L1 menu in HLT relval and Prompt Like GTs #25069

Merged

cmsbuild added tests-pending and removed tests-started labels Oct 31, 2018

revert the hlt_relval and prompt-like GTs, adding new keys for HI hlt…

4db9cf2

…_relval and prompt-like GTs, update the HI L1 menu for HI 2018 upgrade realistic GT

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 31, 2018

cmsbuild added tests-started and removed tests-pending labels Oct 31, 2018

cmsbuild added tests-approved and removed tests-started labels Oct 31, 2018

cmsbuild added comparison-available and removed comparison-pending labels Oct 31, 2018

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Oct 31, 2018

cmsbuild added orp-approved and removed orp-pending labels Oct 31, 2018

cmsbuild merged commit 3033d5c into cms-sw:master Oct 31, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update L1 menu in HLT relval and Prompt Like GTs #25068

Update L1 menu in HLT relval and Prompt Like GTs #25068

tocheng commented Oct 31, 2018 •

edited

cmsbuild commented Oct 31, 2018

cmsbuild commented Oct 31, 2018

cmsbuild commented Oct 31, 2018

tocheng commented Oct 31, 2018

cmsbuild commented Oct 31, 2018 •

edited

Martin-Grunewald commented Oct 31, 2018

tocheng commented Oct 31, 2018

Martin-Grunewald commented Oct 31, 2018 •

edited

Martin-Grunewald commented Oct 31, 2018 •

edited

tocheng commented Oct 31, 2018

Martin-Grunewald commented Oct 31, 2018

tocheng commented Oct 31, 2018

tocheng commented Oct 31, 2018

cmsbuild commented Oct 31, 2018

tocheng commented Oct 31, 2018

cmsbuild commented Oct 31, 2018

cmsbuild commented Oct 31, 2018

Martin-Grunewald commented Oct 31, 2018

tocheng commented Oct 31, 2018

cmsbuild commented Oct 31, 2018 •

edited

cmsbuild commented Oct 31, 2018

cmsbuild commented Oct 31, 2018

cmsbuild commented Oct 31, 2018

fabiocos commented Oct 31, 2018

tocheng commented Oct 31, 2018

cmsbuild commented Oct 31, 2018

fabiocos commented Oct 31, 2018

tocheng commented Oct 31, 2018 •

edited

fabiocos commented Oct 31, 2018

Update L1 menu in HLT relval and Prompt Like GTs #25068

Update L1 menu in HLT relval and Prompt Like GTs #25068

Conversation

tocheng commented Oct 31, 2018 • edited

cmsbuild commented Oct 31, 2018

cmsbuild commented Oct 31, 2018

cmsbuild commented Oct 31, 2018

tocheng commented Oct 31, 2018

cmsbuild commented Oct 31, 2018 • edited

Martin-Grunewald commented Oct 31, 2018

tocheng commented Oct 31, 2018

Martin-Grunewald commented Oct 31, 2018 • edited

Martin-Grunewald commented Oct 31, 2018 • edited

tocheng commented Oct 31, 2018

Martin-Grunewald commented Oct 31, 2018

tocheng commented Oct 31, 2018

tocheng commented Oct 31, 2018

cmsbuild commented Oct 31, 2018

tocheng commented Oct 31, 2018

cmsbuild commented Oct 31, 2018

cmsbuild commented Oct 31, 2018

Martin-Grunewald commented Oct 31, 2018

tocheng commented Oct 31, 2018

cmsbuild commented Oct 31, 2018 • edited

cmsbuild commented Oct 31, 2018

cmsbuild commented Oct 31, 2018

cmsbuild commented Oct 31, 2018

fabiocos commented Oct 31, 2018

tocheng commented Oct 31, 2018

cmsbuild commented Oct 31, 2018

fabiocos commented Oct 31, 2018

tocheng commented Oct 31, 2018 • edited

fabiocos commented Oct 31, 2018

tocheng commented Oct 31, 2018 •

edited

cmsbuild commented Oct 31, 2018 •

edited

Martin-Grunewald commented Oct 31, 2018 •

edited

Martin-Grunewald commented Oct 31, 2018 •

edited

cmsbuild commented Oct 31, 2018 •

edited

tocheng commented Oct 31, 2018 •

edited