Cleanups to DQMStore

[knoepfel]

As discussed in PR #22949, this PR contains only cleanups, and no functional changes--i.e. there are no underlying changes to the DQMStore representation.

There should be no breaking changes to user code or workflows.

[cmsbuild]

The code-checks are being triggered in jenkins.

[knoepfel]

@schneiml, @Dr15Jones FYI

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-23562/5152

[cmsbuild]

A new Pull Request was created by @knoepfel (Kyle Knoepfel) for master.

It involves the following packages:

DQMServices/Core

@vazzolini, @kmaeshima, @dmitrijus, @cmsbuild, @jfernan2, @vanbesien can you please review it and eventually sign? Thanks.
@barvic this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

[Dr15Jones]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/28629/console Started: 2018/06/11 19:51

[cmsbuild]

+1
Tested at: 2953f35
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23562/28629/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23562/28629/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 31
• DQMHistoTests: Total histograms compared: 2901984
• DQMHistoTests: Total failures: 2
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2901792
• DQMHistoTests: Total skipped: 190
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 30 files compared)
• Checked 128 log files, 14 edm output root files, 31 DQM output files

[schneiml]

Thanks @knoepfel!

For the functional changes, we should maybe have a chat/meeting at some point. Our plans on how to restructure DQMStore are getting more concrete recently, and will make some of the IGetter/IBooker changes obsolete: The idea is to break down the DQMStore into one object/container per module (this is easier than it seems, since the old "threaded mode" did exactly that), and then allow "importing" other MEs using proper dependencies (not sure yet what this will look like, but I am pretty sure this is how it should be done). The hard part is to get the default setup with all the harvesters that don't declare dependencies to work (but that is possible, it used to work in threaded DQM). FInally, we will have "legacy modules" once again, but hopefully we can fix them all this time...

[knoepfel]

Yes, @schneiml, having a chat would be good. Next week is looking pretty open for me. If it works well for you, let me know which days/times are good to meet, and we can set up a meeting.

[jfernan2]

Hi @knoepfel @schneiml was this finally discussed? thanks

[schneiml]

@jfernan2 we did talk a bit at CHEP, but not specifically about this. Actually, I think we can go ahead with this PR -- early 10_3 cycle is also a perfect time for such things.

[jfernan2]

please test
OK, let's test it again

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/29321/console Started: 2018/07/23 11:11

[cmsbuild]

+1
Tested at: 2953f35
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23562/29321/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23562/29321/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 31
• DQMHistoTests: Total histograms compared: 2889291
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2889100
• DQMHistoTests: Total skipped: 190
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 30 files compared)
• Checked 128 log files, 14 edm output root files, 31 DQM output files

[andrius-k]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

[schneiml]

@knoepfel now once we get this in, we should also look at the original PR again. Early 10_3 would again be the right time for bigger changes.

If you are busy with other things, we can do the porting to a new PR -- let us know.

[kpedro88]

+1

[knoepfel]

@schneiml yes, let's look at the overall plan again. I'm on vacation next week, but I'll try to get ahold of you the week of Aug. 6.