Add wfs for JetMET PD

[sam7k9621]

PR description:

Add Run3 skims to the standard relval wfs: 140.119

PR validation:

Should be tested on 140.119

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

None

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39405/32112

• This PR adds an extra 20KB to repository

[cmsbuild]

A new Pull Request was created by @sam7k9621 for master.

It involves the following packages:

• Configuration/PyReleaseValidation (pdmv, upgrade)

@bbilin, @cmsbuild, @AdrianoDee, @srimanob, @kskovpen, @sunilUIET can you please review it and eventually sign? Thanks.
@makortel, @kpedro88, @fabiocos, @Martin-Grunewald, @missirol, @trtomei, @beaucero, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

[srimanob]

@cmsbuild please test workflow 140.119

[cmsbuild]

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1424b1/27576/summary.html
COMMIT: a3c9464
CMSSW: CMSSW_12_6_X_2022-09-15-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39405/27576/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @CTPPS PPS NANOAOD dependency hotfix #39294
• @eyigitba Modify RPC and GEM unpacker blocks in EMTF unpacker to match the new Run 3 format #39388

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1424b1/27576/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1424b1/27576/git-merge-result

RelVals-INPUT

The relvals timed out after 4 hours.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-1424b1/140.119_RunJetMET2022D+RunJetMET2022D+HLTRUN3+RECONANORUN3+SKIMJETMETRUN3

Summary:

• No significant changes to the logs found
• Reco comparison results: 5 differences found in the comparisons
• DQMHistoTests: Total files compared: 51
• DQMHistoTests: Total histograms compared: 3618326
• DQMHistoTests: Total failures: 27
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3618277
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
• Checked 212 log files, 49 edm output root files, 51 DQM output files
• TriggerResults: no differences found

[sam7k9621]

Hi @kskovpen @srimanob
It seems that it fails because of timeout, could we only test on the used wfs instead?
Something like:
workflows = 140.119
relvals_opt = -w standard

[srimanob]

We can just see that the new workflow runs fine,
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1424b1/27576/runTheMatrix-results/140.119_RunJetMET2022D+RunJetMET2022D+HLTRUN3+RECONANORUN3+SKIMJETMETRUN3/

I think limit relvals_opt the workflow will cause issue as some of them will not find the proper relvals_opt.

[srimanob]

test parameters:

• workflow = 140.119
• relvals_opt = --what cleanedupgrade,standard,highstats,pileup,generator,extendedgen,production,ged,machine,premix

[srimanob]

@cmsbuild please test

[sam7k9621]

Hi @srimanob , thank you for the quick reply.
We found that only the EXOHighMET skim could not be found under JetMET PD in RunD (can be found under MET in RunC).
https://cmsweb.cern.ch/das/request?view=list&limit=50&instance=prod%2Fglobal&input=dataset%3D%2F*%2FRun2022*EXOHighMET*%2F*
However other skims seem to be fine (EXODelayedJetMET, JetHTJetPlusHOFilter).
The 140.119 wfs should report the bug, but it seems to produce the JetMET files successfully.
Could you please let me know what might be wrong?

[srimanob]

Hi @sam7k9621

We found that only the EXOHighMET skim could not be found under JetMET PD in RunD (can be found under MET in RunC).

This is because Tier-0 configuration does not contain EXOHighMET
https://cmsweb.cern.ch/t0wmadatasvc/prod/reco_config?run=357538

{
  "run": 357538, 
  "write_miniaod": true, 
  "scram_arch": "el8_amd64_gcc10", 
  "write_aod": true, 
  "write_reco": false, 
  "alca_skim": "HcalCalIsoTrkProducerFilter,TkAlJetHT,HcalCalNoise", 
  "cmssw": "CMSSW_12_4_6", 
  "physics_skim": "EXODelayedJetMET,JetHTJetPlusHOFilter,LogError,LogErrorMonitor", 
  "write_dqm": true, 
  "scenario": "ppEra_Run3", 
  "global_tag": "124X_dataRun3_Prompt_v4", 
  "multicore": 8, 
  "dqm_seq": "@common,@jetmet,@L1TMon,@hcal", 
  "primary_dataset": "JetMET"
}

You need to ask Tier-0 team on what happened.

[sam7k9621]

@srimanob thanks a lot, I will contact with them

[cmsbuild]

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1424b1/27588/summary.html
COMMIT: a3c9464
CMSSW: CMSSW_12_6_X_2022-09-15-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39405/27588/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test test-das-selected-lumis had ERRORS

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-1424b1/140.119_RunJetMET2022D+RunJetMET2022D+HLTRUN3+RECONANORUN3+SKIMJETMETRUN3

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 51
• DQMHistoTests: Total histograms compared: 3618332
• DQMHistoTests: Total failures: 2
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3618308
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
• Checked 212 log files, 49 edm output root files, 51 DQM output files
• TriggerResults: no differences found

[srimanob]

Do we still need this PR, as the issue happened in the Tier-0 configuration side? @sam7k9621