DQM: fix some autoDQM.py harvesting sequences

[schneiml]

PR description:

It turns out that ConfigBuilder treats failing to find a HARVESTING sequence not as an error, and in fact we have some non-existing sequences in autoDQM.py. One of them is DQMHarvestDCS, which I forgot to remove in #28829.

But also DQMNone, which is used to signify that there is nothing to do ('' could be understood as default sequence, as usual in cmsDriver) is not actually defined in harvesting. So I added that to all the main config files.

Finally, this PR makes ConfigBuilder fail on undefined harvesting sequences, like it does for other sequences.

PR validation:

Functionality should not be affected, but misconfigured jobs may crash now. I expect some PR and even IB failures untilwe catch all of them.

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29343/14432

• This PR adds an extra 44KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File DQMOffline/Configuration/python/DQMOffline_SecondStep_cff.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983

[cmsbuild]

A new Pull Request was created by @schneiml (Marcel Schneider) for master.

It involves the following packages:

Configuration/Applications
DQMOffline/Configuration

@andrius-k, @kmaeshima, @schneiml, @cmsbuild, @silviodonato, @franzoni, @jfernan2, @fioriNTU, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @threus, @fabiocos, @rociovilar this is something you requested to watch as well.
@davidlange6, @silviodonato, @fabiocos you are the release manager for this.

cms-bot commands are listed here

[schneiml]

please test

This has some chance of blowing up due to more undefined harvesting sequences. Let's see.

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/5455/console Started: 2020/03/31 11:57

[cmsbuild]

+1
Tested at: fc40c3c
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e4e904/5455/summary.html
CMSSW: CMSSW_11_1_X_2020-03-30-2300
SCRAM_ARCH: slc7_amd64_gcc820

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e4e904/5455/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 18 differences found in the comparisons
• DQMHistoTests: Total files compared: 34
• DQMHistoTests: Total histograms compared: 2692110
• DQMHistoTests: Total failures: 40
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2691751
• DQMHistoTests: Total skipped: 319
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
• Checked 147 log files, 16 edm output root files, 34 DQM output files

[schneiml]

+1

looks good so far, though be aware of potential failures due to misconfigured harvesting in the IB tests.
Edit: actually, now that I think of it, the new PR tests from #29150 should have triggered all possible crashes in the unit tests, and there were none.

[silviodonato]

+operations

[silviodonato]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

[silviodonato]

@schneiml we are getting errors in CMSSW_11_1_X_2020-04-02-1100 in wf 140.0, 145.0, 150.0 .
https://cmssdt.cern.ch/SDT/cgi-bin/buildlogs/raw/slc7_amd64_gcc820/CMSSW_11_1_X_2020-04-02-1100/pyRelValMatrixLogs/run/145.0_HydjetQ_B12_5020GeV_2015+HydjetQ_B12_5020GeV_2015+DIGIHI2015+RECOHI2015+HARVESTHI2015/step4_HydjetQ_B12_5020GeV_2015+HydjetQ_B12_5020GeV_2015+DIGIHI2015+RECOHI2015+HARVESTHI2015.log

Do you think it is related to this PR? Tomorrow I will check it better.

[schneiml]

@silviodonato yep, that is related. validationHarvestingNoHLT (in your link) does not exist and should be removed from all configs that use it.

[silviodonato]

Do you mean we have to remove validationHarvestingNoHLT+ from HARVESTHI2011, HARVESTHI2015, HARVESTHI2018 in
https://github.com/cms-sw/cmssw/blob/master/Configuration/PyReleaseValidation/python/relval_steps.py
?
pdmv: @chayanit @pgunnell @wajidalikhan

[silviodonato]

validationHarvestingNoHLT is defined here https://github.com/cms-sw/cmssw/blob/master/Configuration/StandardSequences/python/Harvesting_cff.py#L26

[silviodonato]

#29393 should fix the error