Phase2 improved conditions and ntuple maker update

[OzAmram]

This PR updates the phase2 template and genError objects to have extended angle coverage. This was needed in order to correct a previous issue where there were a good number of hits outside their angle space and lead to poor resolution.
Results validating these condition changes were shown at the phase 2 simulation meeting here

It also has an update to the Phase 2 ntuple maker used in validation studies. This update saves more information about the track the hits belong to and the local angles used in the hit reconstruction. Because the ntuple maker now requires the trajectory as an input configuration files that use it must be changed, the example configuration file in the same directory has been updated to reflect this.

@mmusich @tsusa @emiglior @mtosi @tvami @pmaksim1 @jalimena @perrotta @slava77

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32220/19967

• This PR adds an extra 48KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File RecoLocalTracker/SiPixelRecHits/interface/PixelCPEBase.h modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, Patatrack integration - Pixel local reconstruction (9/N) #31721

[cmsbuild]

A new Pull Request was created by @OzAmram (Oz Amram) for master.

It involves the following packages:

CondFormats/SiPixelTransient
Configuration/AlCa
RecoLocalTracker/SiPixelRecHits
SLHCUpgradeSimulations/Geometry

@perrotta, @civanch, @Dr15Jones, @makortel, @cvuosalo, @tlampen, @christopheralanwest, @ianna, @mdhildreth, @cmsbuild, @jpata, @yuanchao, @tocheng, @slava77, @ggovi, @pohsun, @kpedro88 can you please review it and eventually sign? Thanks.
@fabiocos, @makortel, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @Martin-Grunewald, @gpetruc, @dkotlins, @tocheng, @ferencek, @mtosi, @mmusich, @threus, @ebrondol, @seemasharmafnal, @tvami this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[cvuosalo]

@cmsbuild please test

[cmsbuild]

The tests are being triggered in jenkins.

• CMSSW_11_2_X_2020-11-20-1100/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10913/console Started: 2020/11/20 21:16

[mmusich]

@cvuosalo I am expecting this round of tests to fail. An update of SLHCUpgradeSimulations/Geometry/test/phase2_digi_reco_pixelntuple_cfg.py is needed.

[cvuosalo]

The changes to this comment look wrong. "Extended2023D41" does not exist anymore. And what is "no_usmcsexec"?

[OzAmram]

Sorry I think this is left over from some very old code. I updated the comment

[cmsbuild]

-1

Tested at: d90c0f0

CMSSW: CMSSW_11_2_X_2020-11-20-1100
SCRAM_ARCH: slc7_amd64_gcc820
You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7d7586/10913/summary.html

I found follow errors while testing this PR

Failed tests: UnitTests

• Unit Tests:

I found errors in the following unit tests:

---> test testPhase2PixelNtuple had ERRORS

[cmsbuild]

Comparison job queued.

[OzAmram]

I just pushed a new commit that updated SLHCUpgradeSimulations/Geometry/test/phase2_digi_reco_pixelntuple_cfg.py and should fix the failed test.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7d7586/11164/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 3375 differences found in the comparisons
• DQMHistoTests: Total files compared: 35
• DQMHistoTests: Total histograms compared: 2529593
• DQMHistoTests: Total failures: 33833
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 2495737
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.004 KiB( 34 files compared)
• DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
• Checked 148 log files, 34 edm output root files, 35 DQM output files

[kpedro88]

Comparison differences are limited to Phase 2 workflows. Reconstructed tracks see minor changes, which propagate downstream:

[kpedro88]

+upgrade

[makortel]

Yes I think the warning collector clearly needs to be reworked a bit and maybe it should be discussed more exactly what behavior we want.

(this is probably a wrong place to continue discussion) One option would be to utilize MessageLogger itself. If each of the "unique message" would have unique category as well, in principle MessageLogger could be configured to limit the number messages shown for those categories (e.g. to 1). MessageLogger still counts all the messages of that category, and those counts can e.g. be put into the Event and propagated to DQM. This option would need some solution for #32161 (comment).

[cvuosalo]

+1

[mmusich]

@perrotta, @christopheralanwest, @jpata, @yuanchao, @slava77, @ggovi,
could you please let us know if you have other comments on this PR and eventually sign?
Thanks

[slava77]

+1

for #32220 f3c5e30

• code changes are in line with the PR description and the follow up review; the most recent update now reduced the changes in reco to just the update in CondFormats/SiPixelTransient, where the GenError inputs for phase2 were updated
• jenkins tests pass and comparisons with the baseline show (relevant) differences only in the phase-2 workflow; the differences at the level of tracks are generally small

[silviodonato]

kind reminder @cms-sw/alca-l2 @ggovi

[yuanchao]

+1

• no errors/warnings on static checks; python configurations build ok
• use of messagelogger
• gvz is actually 1/gvz which is not very straight forward...

[qliphy]

kind reminder @cms-sw/db-l2

[silviodonato]

merge
@ggovi the DB changes seem trivial ( https://github.com/cms-sw/cmssw/pull/32220/files?diff=split&w=1#diff-056af923e725c77ed7309805ab3f02f244be646b0c50e496ce9c14fe2f65310aL43-L45 )
please let us know if you have comments