New muon validation fixes

[abbiendi]

These are fixes to the New muon validation, determined after checks of few relevant workflows.
Comparison plots of New vs production validation are visible here:
https://cmsdoc.cern.ch/cms/Physics/muon/CMSSW/Performance/RecoMuon/Validation/val/test_NewValidation/
All the visible features are expected, due to the new validation.
In addition it synchronizes the New validation with concurrent changes in the production version.

[cmsbuild]

A new Pull Request was created by @abbiendi for master.

It involves the following packages:

Validation/RecoMuon

@cmsbuild, @dmitrijus, @vanbesien, @davidlange6 can you please review it and eventually sign? Thanks.
@battibass, @jhgoh, @calderona, @HuguesBrun, @trocino, @rociovilar this is something you requested to watch as well.
@Muzaffar, @davidlange6, @smuzaffar you are the release manager for this.

cms-bot commands are listed here #13028

[abbiendi]

Few more details.
The tested workflows should cover all the possible cases. They are:

• ZMM 13 TeV no PU (1330.0)
• ZMM 13 TeV with PU25ns (25206.0)
• ZEEMM_13 Heavy Ions (140.4)
• TTbar 13 TeV FastSim (135.1)
• Phase2 Single Mu pt100 "2023D4Timing" (22808.0)
  To produce them with the New validation activated (in place of the current one):
  runTheMatrix.py -l 1330.0,25206.0,140.4,135.1 --command="--customise=Validation/RecoMuon/customiseForNewMuonValidation.enableNewMuonVal"
  runTheMatrix.py -w upgrade -l 22808.0 --command="--customise=Validation/RecoMuon/customiseForNewMuonValidation.enableNewMuonVal"
  These is the minimal set of workflows that should be tested before switching to the New validation. Probably a few more could be added (as a single muon gun in Run2 configuration).

[dmitrijus]

+1

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/19101/console Started: 2017/04/11 15:30

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs after it passes the integration tests. This pull request requires discussion in the ORP meeting before it's merged. @Muzaffar, @davidlange6, @smuzaffar

[abbiendi]

just to make it clear: this PR is necessary to move on with the agreed plan towards the transition to the New validation.
After it is merged in 91X we will ask for backport in 90X to finalize the benchmark comparisons in a stable release.
The comparison with the Old (current) will need to activate the New validation in a few RelVal workflows (min set >= those tested here).

[cmsbuild]

+1
Tested at: 1166107
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18290/19101/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18290/19101/summary.html

Comparison Summary:

• You potentially added 15 lines to the logs
• Reco comparison results: 1647 differences found in the comparisons
• DQMHistoTests: Total files compared: 23
• DQMHistoTests: Total histograms compared: 1921141
• DQMHistoTests: Total failures: 52696
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 1868272
• DQMHistoTests: Total skipped: 173
• DQMHistoTests: Total Missing objects: 0
• Checked 94 log files, 14 edm output root files, 23 DQM output files

[abbiendi]

Hi, about the above comparison summary: there cannot be any sensible difference if (as I assume) the tested workflows are all standard ones, given that my changes here concern only the New muon validation, which has to be enabled by a customization.

[davidlange6]

+1
(though no plans to back port this in its current form)