Fix avoiding dqmofflineOnPAT_step duplications

[jfernan2]

PR description:

Fix to avoid duplicated sequences with dqmOffline_step for cmsDriver DQM:selections not defined in autoDQM or without PostDQM sequence

PR validation:

Tested with runTheMatrix.py -l all -i all --ibeos

if this PR is a backport please specify the original PR:

No backport expected

[jfernan2]

please test

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28113/12134

• This PR adds an extra 36KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/Applications/python/ConfigBuilder.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2782/console Started: 2019/10/04 13:28

[cmsbuild]

A new Pull Request was created by @jfernan2 for master.

It involves the following packages:

Configuration/Applications

@cmsbuild, @franzoni, @fabiocos, @kpedro88, @davidlange6 can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

[cmsbuild]

+1
Tested at: a46d539
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a34663/2782/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a34663/2782/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 14 differences found in the comparisons
• DQMHistoTests: Total files compared: 34
• DQMHistoTests: Total histograms compared: 2927797
• DQMHistoTests: Total failures: 129
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2927327
• DQMHistoTests: Total skipped: 341
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -59134.872 KiB( 33 files compared)
• DQMHistoSizes: changed ( 250202.181,... ): -640.710 KiB JetMET/PFCandidates
• DQMHistoSizes: changed ( 250202.181,... ): -448.898 KiB ParticleFlow/slimmedJetValidation
• DQMHistoSizes: changed ( 250202.181,... ): -263.359 KiB ParticleFlow/slimmedElectronValidation
• DQMHistoSizes: changed ( 250202.181,... ): -261.354 KiB ParticleFlow/SlimmedMuonValidation
• DQMHistoSizes: changed ( 250202.181,... ): -255.191 KiB JetMET/Jet
• DQMHistoSizes: changed ( 250202.181,... ): -239.348 KiB ParticleFlow/slimmedMETValidation
• DQMHistoSizes: changed ( 250202.181,... ): -239.010 KiB Physics/Top
• DQMHistoSizes: changed ( 250202.181,... ): -200.197 KiB JetMET/MET
• DQMHistoSizes: changed ( 250202.181,... ): -37.338 KiB ParticleFlow/PFJetResValidation
• DQMHistoSizes: changed ( 250202.181,... ): -24.309 KiB Tracking/PackedCandidate
• DQMHistoSizes: changed ( 22034.0 ): ...
• Checked 147 log files, 16 edm output root files, 34 DQM output files

[jfernan2]

hold
This change needs deeper research...

[cmsbuild]

Pull request has been put on hold by @jfernan2
They need to issue an unhold command to remove the hold state or L1 can unhold it for all

[fabiocos]

@jfernan2 I am afraid the changed in wf 11634.0 are not related to this PR but to some non reproducibility around since a week or so.

[cmsbuild]

The code-checks are being triggered in jenkins.

[jfernan2]

please test

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28113/12162

• This PR adds an extra 40KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/Applications/python/ConfigBuilder.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2824/console Started: 2019/10/07 19:16

[cmsbuild]

Pull request #28113 was updated. @cmsbuild, @franzoni, @fabiocos, @kpedro88, @davidlange6 can you please check and sign again.

[cmsbuild]

+1
Tested at: 2b3440b
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a34663/2824/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a34663/2824/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 34
• DQMHistoTests: Total histograms compared: 2961052
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2960710
• DQMHistoTests: Total skipped: 341
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
• Checked 147 log files, 16 edm output root files, 34 DQM output files

[fabiocos]

@jfernan2 could you please point me to a pattern that would lead to the problem removed by this PR? Which test workflow is showing it?

[jfernan2]

@fabiocos yes, this was spotted by @missirol duplication occurs for example in customized settings like:

cmsDriver.py step1 --conditions auto:phase1_2018_realistic --customise Configuration/DataProcessing/Utils.addMonitoring --datatier DQMIO --era Run2_2018 --eventcontent DQM --filein /store/relval/CMSSW_10_6_1/RelValTTbarLepton_13UP18/FEVTDEBUGHLT/PUpmx25ns_106X_upgrade2018_realistic_v6_ul18hlt_premix_hs-v1/20000/1A05F6A1-AE1B-724C-B9E2-A081DE805133.root --geometry DB:Extended --mc --nThreads 1 --runUnscheduled --step DQM:offlineHLTSourceOnAOD -n 100

[fabiocos]

+operations

the use case mentioned by @jfernan2 is fixed, while the normal workflows look unaffected

[fabiocos]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.