DQM OuterTrackerL1 plots for TTStubs, TTClusters, and TTTracks for release validation

[emacdonald16]

Hello,
I modified the DQM/Phase2OuterTracker package (including a name change to OuterTrackerL1) to include plots for L1 tracks, and changed some of the plots in the stubs and clusters, as well. I added resolution and efficiency plots for the L1 tracks. Since these include MC information, I have added a new package, Validation/OuterTrackerL1V, to contain these. I modified the DQMOffline/Configuration and Validation/Configuration packages slightly, to include the running of these packages in runTheMatrix.py. Here are links to a RelMon comparison for 6600 ttbar no PU events, so that you can see all the plots for the DQM package (1) and the Validation package (2). The comparisons in these webpages are meaningless, since they are comparing one sample to itself, but the webpages allow you to look through the plots I propose be added.

1: http://www-hep.colorado.edu/~emacdonald16/FinalDQM/Phase2OuterTracker.html
2: http://www-hep.colorado.edu/~emacdonald16/FinalDQM/Phase2OuterTrackerV.html

I will also be monitoring these plots once they are included in the sequence.

[emacdonald16]

@boudoul FYI

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-22682/4051

Code check has found code style and quality issues which could be resolved by applying a patch in https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-22682/4051/git-diff.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-22682/4051/git-diff.patch | patch -p1

You can run scram build code-checks to apply code checks directly

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-22682/4059

[cmsbuild]

A new Pull Request was created by @emacdonald16 for master.

It involves the following packages:

DQM/OuterTrackerL1
DQM/Phase2OuterTracker
DQMOffline/Configuration
Validation/Configuration
Validation/OuterTrackerL1V

The following packages do not have a category, yet:

DQM/OuterTrackerL1
Validation/OuterTrackerL1V
Please create a PR for https://github.com/cms-sw/cms-bot/blob/master/categories_map.py to assign category

@perrotta, @civanch, @vazzolini, @kmaeshima, @mdhildreth, @dmitrijus, @cmsbuild, @jfernan2, @slava77, @vanbesien can you please review it and eventually sign? Thanks.
@threus, @swertz, @rovere, @rociovilar this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

[cmsbuild]

+1
Tested at: 05717d3
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22682/27427/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22682/27427/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 29
• DQMHistoTests: Total histograms compared: 2504338
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2504161
• DQMHistoTests: Total skipped: 176
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.599999999882 KiB( 20 files compared)
• Checked 119 log files, 9 edm output root files, 29 DQM output files

[kpedro88]

+1

[boudoul]

Dear @civanch , could you please re-sign this PR ? thank you

[civanch]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

[fabiocos]

@emacdonald16 apparently there is no difference in the output of the test, in which test workflows is this code supposed to be run?

[fabiocos]

@emacdonald16 @boudoul ok, in the phase 2 wf 21234 about 23 MB are added:

new:
24946.52 KiB Phase2OuterTracker/Stubs
24872.20 KiB Phase2OuterTracker/Clusters

old:
14101.61 KiB SiOuterTracker/Stubs
14027.34 KiB SiOuterTracker/Clusters

old: Total bytes: 703537.53 KiB
new: Total bytes: 725186.02 KiB

While the phase2 DQM is much larger than the 2018 production one, this is nevertheless a significant increase. Do you see any possible optimisation?

[boudoul]

Hi @fabiocos , just to log here what we just discussed in person , I'm not really surprised by an increase since we are monitoring new objects for phase2 - However we may want to indeed take a look on how to optimize (if not this developments) sone other phase2 tk parts .

[emacdonald16]

@fabiocos @boudoul The new packages should be included in any Phase 2 workflow that isn't just tracking only, so 20008.0, 20034.0, etc. We can also look into optimization for some of the parts, as Gaelle said.

[fabiocos]

@boudoul @emacdonald16 how do we proceed? Do you want to work more on this PR, or are you planning to come with some optimisation in a second stage?

[emacdonald16]

@fabiocos I think optimization in a second stage would be a better idea, preferably after L1 tracking has been added. That way, we can optimize the tracks at the same time as the clusters and stubs. @boudoul do you agree?

[boudoul]

Right ; between my night shifts and CMS week, I'm afraid I won't be able to look at this before at least a week - But it should be followed up in a second stage

[fabiocos]

+1