Enable all working DQM and validation modules for 2017

[makortel]

This PR attempts to enable all DQM and validation modules that work out of the box for 2017. Exceptions are

• Tracking DQM (needs to be migrated to the new iterations; will be done after the integration new tracking for 2017)
• Pixel DQM and validation
• HLT DQM and validation as HLT is currently not run as part of 2017 workflows
  • Also modules not working or printing excessively because of missing HLT are not enabled

Following @deguio's comment #12806 (comment), the already-era-migrated DQM/validation sequence customizations are moved back to phase1TkCustoms.py in order to treat them all consistently.

Tested in CMSSW_8_1_X_2016-02-29-2300.

[cmsbuild]

A new Pull Request was created by @makortel (Matti Kortelainen) for CMSSW_8_1_X.

It involves the following packages:

DQMOffline/Configuration
DQMOffline/Muon
HLTriggerOffline/Common
SLHCUpgradeSimulations/Configuration
Validation/RecoTrack

@civanch, @mdhildreth, @cmsbuild, @deguio, @vanbesien, @davidlange6 can you please review it and eventually sign? Thanks.
@barvic, @battibass, @threus, @abbiendi, @GiacomoSguazzoni, @jhgoh, @VinInn, @Martin-Grunewald, @bellan, @istaslis, @rovere, @wmtford, @cerati, @trocino, @dgulhan, @rociovilar this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

[makortel]

@deguio @vanbesien @davidlange6 Despite what I did here I think we should reconsider the the strategy on where these DQM/VALIDATION/HARVESTING (temporary) customizations for 2017 workflows should be made. I moved the era-migrated ones back to phase1TkCustoms because @deguio was earlier against having them scattered around (#12806 (comment)). On the other hand, once the followup of #13477 (which I have ready in a private branch just waiting #13477 to be merged) these will be the only customizations left in phase1TkCustoms and thus the only piece preventing its deletion. I guess the HLT-related modules would be last to be enabled (thus defining the required lifetime for these customizations).

So I think the question is on the balance between

• the place where "not-yet-working" DQM modules should be removed from the sequences
• how soon to clean up the customize function

Personally I would be in favour of removing phase1TkCustoms as soon as possible (but I may be too eager), or at least cleaning up the already-migrated parts.

[boudoul]

Big thank Matti, I put @fioriNTU here so that he can follow the process too

[schneiml]

Is there a clean way to locally test a branch like this?

When I checkout this branch (from CMS_8_0_0) and git cms-addpkg the involved packages, runTheMatrix.py --what upgrade -l 10000 --command "-n 1" fails in step2, complaining about HcalGains. Earlier (on something based on 8_0_0_pre6, but not clean), it was missing files related to Hcal...

[makortel]

@schneiml 2017 workflow is broken in 800, but it works again in 800patch1.

For this specific branch you need to take the same IB the branch is based on, or later (I typically put the IB on the description for my own bookkeeping because I'm not aware of a way to find it easily in GitHub), and then git cms-merge-topic 13590. If you take anything earlier than the base IB, the merge/checkout will pull all changes between the developer area release and the base IB. Also checking out only the involved packages is not generally enough as dependent packages may need to be recompiled (git cms-checkdeps -a adds all dependent packages, and git cms-merge-topic does that too).

[VinInn]

@cmsbuild , please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/11733/console

[cmsbuild]

+1
Tested at: 5ca659e
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13590/11733/summary.html

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13590/11733/summary.html

[schneiml]

@makortel git cms-merge-topic is what I was looking for, thanks for the hint!

[civanch]

+1

[deguio]

+1
thanks @makortel . we can probably talk in person. I am not against the removal of the phase1TkCustoms. my original comment was done in the early stages of the discussion on eras.
F.

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_8_1_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @slava77, @davidlange6, @Degano, @smuzaffar

[makortel]

Is there anything holding back the integration of this PR?

[davidlange6]

+1