[93X] Improvements to L1 Offline DQM - part 2 (issue #19592)

[kreczko]

This PR implements the improvements requested in issue #19592 by the L1T team.

@tapperad et all, please have a look

[cmsbuild]

A new Pull Request was created by @kreczko (Luke Kreczko) for master.

It involves the following packages:

DQMOffline/L1Trigger

@vazzolini, @kmaeshima, @dmitrijus, @cmsbuild, @rekovic, @vanbesien, @mulhearn, @davidlange6 can you please review it and eventually sign? Thanks.
@thomreis, @rociovilar this is something you requested to watch as well.
@davidlange6 you are the release manager for this.

cms-bot commands are listed here

[dmitrijus]

+1

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/21578/console Started: 2017/07/18 16:30

[cmsbuild]

-1

Tested at: 65a24fc

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
f56dad1
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19796/21578/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19796/21578/git-merge-result

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19796/21578/summary.html

I found follow errors while testing this PR

Failed tests: UnitTests

• Unit Tests:

I found errors in the following unit tests:

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
f56dad1
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19796/21578/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19796/21578/git-merge-result

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19796/21578/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 23
• DQMHistoTests: Total histograms compared: 2025409
• DQMHistoTests: Total failures: 15040
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2010203
• DQMHistoTests: Total skipped: 166
• DQMHistoTests: Total Missing objects: 0
• Checked 93 log files, 14 edm output root files, 23 DQM output files

[kreczko]

As far as I can see all unit tests succeded (searching for ---> test in the unit test log).
The only error message in the logs is

dropped waiting message count 0
stat: cannot stat `dqm_empty.root': No such file or directory
read_missing_file_cfg.py ------------------------------------------------------------
18-Jul-2017 16:36:46 CEST  Initiating request to open file file:dqm_missing.root
18-Jul-2017 16:36:47 CEST  Successfully opened file file:dqm_missing.root
----- Begin Fatal Exception 18-Jul-2017 16:36:47 CEST-----------------------
An exception of category 'FileReadError' occurred while
   [0] Calling InputSource::readFile_
   [1] Opening DQM Root file
Exception Message:
Input file file:dqm_missing.root appears to be corrupted since it does not have the proper internal structure.
 Check to see if the file was closed properly.
----- End Fatal Exception -------------------------------------------------

If I understand from the name, this file should not exist (and the test segment is marked as success).

What am I missing?

[dmitrijus]

The test failure is unrelated to this PR. I will look into it.

[kreczko]

@dmitrijus Did you have any time to look into this?

[dmitrijus]

please test

[cmsbuild]

Pull request #19796 was updated. @vazzolini, @kmaeshima, @dmitrijus, @cmsbuild, @rekovic, @vanbesien, @mulhearn can you please check and sign again.

[dmitrijus]

+1

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/22174/console Started: 2017/08/09 18:06

[cmsbuild]

+1
Tested at: f362690
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19796/22174/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
f362690
5d342d6
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19796/22174/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19796/22174/git-merge-result

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19796/22174/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /build/cmsbld/jenkins-workarea/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-19796/11624.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2019_GenSimFull+DigiFull_2019+RecoFull_2019+ALCAFull_2019+HARVESTFull_2019

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 26
• DQMHistoTests: Total histograms compared: 2868181
• DQMHistoTests: Total failures: 1051
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2866941
• DQMHistoTests: Total skipped: 189
• DQMHistoTests: Total Missing objects: 0
• Checked 107 log files, 14 edm output root files, 26 DQM output files

[kreczko]

@rekovic @mulhearn
Tests are now passing, only the L1T signature is missing.

[rekovic]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar (and backports should be raised in the release meeting by the corresponding L2)

[davidlange6]

+1