PPS: alignment from auto GT

[jan-kaspar]

PR description:

The default source of PPS alignment is changed to CondDB via auto GT.

The only two changes wrt. master are:

• c3302ac: cherry-pick of @tocheng's 2490f47 where auto GTs are updated
• 2008837: update to PPS configuration to use the alignment from DB

This PR is a replacement for the earlier-closed #26243 .

PR validation:

Below a comparison of results of 2016 data reconstruction (file /store/data/Run2016C/BTagCSV/AOD/07Aug17-v1/110000/0026FCD2-369A-E711-920C-0025905A607E.root). Blue histogram: master with alignment from XML files, red: this PR. There is perfect agreement.

Below a check on pixel reconstruction quality, re-reco performed on file /store/data/Run2017C/ZeroBias/RAW/v1/000/301/283/00000/8ED63519-2282-E711-9073-02163E01A3C6.root. The quality is maintained.

[cmsbuild]

The code-checks are being triggered in jenkins.

[jan-kaspar]

FYI @fabferro @forthommel @tocheng @clemencia @malbouis

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-26305/9004

• This PR adds an extra 16KB to repository

[cmsbuild]

A new Pull Request was created by @jan-kaspar for master.

It involves the following packages:

CalibPPS/ESProducers
Configuration/AlCa
RecoCTPPS/ProtonReconstruction
Validation/CTPPS

The following packages do not have a category, yet:

CalibPPS/ESProducers
Please create a PR for https://github.com/cms-sw/cms-bot/blob/master/categories_map.py to assign category

@perrotta, @andrius-k, @kmaeshima, @schneiml, @tocheng, @cmsbuild, @franzoni, @jfernan2, @fioriNTU, @slava77, @pohsun can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @forthommel, @mmusich, @tocheng this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

[jan-kaspar]

I have run AddOn tests on LXPLUS7: 47 tests passed, 0 failed.

Could some trigger tests, please?

[perrotta]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/33870/console Started: 2019/04/01 14:07

[slava77]

The following packages do not have a category, yet:

CalibPPS/ESProducers

I created cms-sw/cms-bot#1106

[cmsbuild]

+1
Tested at: 2008837
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26305/33870/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
ac9af74
915521e
e076003
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26305/33870/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26305/33870/git-merge-result

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26305/33870/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 8281 differences found in the comparisons
• DQMHistoTests: Total files compared: 32
• DQMHistoTests: Total histograms compared: 3139747
• DQMHistoTests: Total failures: 14820
• DQMHistoTests: Total nulls: 2
• DQMHistoTests: Total successes: 3124728
• DQMHistoTests: Total skipped: 197
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.891 KiB( 31 files compared)
• DQMHistoSizes: changed ( 7.3 ): 0.831 KiB SiStrip/MechanicalView
• DQMHistoSizes: changed ( 7.3 ): -0.004 KiB MessageLogger/Warnings
• DQMHistoSizes: changed ( 250202.181 ): 0.064 KiB SiStrip/MechanicalView
• Checked 133 log files, 14 edm output root files, 32 DQM output files

[slava77]

@cmsbuild please test

to get cleaner comparisons

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/33913/console Started: 2019/04/02 17:40

[cmsbuild]

+1
Tested at: 2008837
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26305/33913/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26305/33913/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 70 differences found in the comparisons
• DQMHistoTests: Total files compared: 32
• DQMHistoTests: Total histograms compared: 3139747
• DQMHistoTests: Total failures: 14
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3139536
• DQMHistoTests: Total skipped: 197
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 31 files compared)
• Checked 133 log files, 14 edm output root files, 32 DQM output files

[slava77]

I have a question about the failure comparison from the test.
For e.g., the results from workflow 136.731,
Is the difference just due to statistics or some other reasons?

The master branch only contains internal alignment of pixel RPs. With this PR a more complete set of alignment corrections is used. Therefore the difference. For 2016 data (used in the test you mention) the alignment in DB is a good candidate for the UL re-reco.

I see also a change in 136.85 (Run2018A , run 315489; red is with this PR)

Is this also expected?

[jan-kaspar]

I see also a change in 136.85 (Run2018A , run 315489; red is with this PR)
...
Is this also expected?

To me it makes sense. The XML file currently in the release, by mistake, does not contain alignment for 2018. This alignment however has been uploaded in DB and it used with this PR. Therefore I am not surprised to see the increase of hits contributing to track reconstruction. Nevertheless, @robutti could you please check and confirm? Thanks in advance!

[slava77]

+1

for #26305 2008837

• code changes are in line with the PR description
• jenkins tests pass and comparisons with the baseline show differences only in PPS in agreement with the changes between the XML and GT setup

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

[fabiocos]

+1

[robutti]

Hi Jan, sorry, I have been able to reply only now. It seems to me this is not needed any more; however, if we are really comparing tracks built without and with internal alignment, isn’t a one-entry difference too little? How large are the samples used for the checks? Thanks, ciao. Enrico Inviato da iPad Il giorno 3 apr 2019, alle ore 11:46, jan-kaspar <notifications@github.com<mailto:notifications@github.com>> ha scritto: I see also a change in 136.85 (Run2018A , run 315489; red is with this PR) ... Is this also expected? To me it makes sense. The XML file currently in the release, by mistake, does not contain alignment for 2018. This alignment however has been uploaded in DB and it used with this PR. Therefore I am not surprised to see the increase of hits contributing to track reconstruction. Nevertheless, @robutti<https://github.com/robutti> could you please check and confirm? Thanks in advance! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#26305 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AFF1u0pAGEHGcHV5iO278rwB2xT4AM99ks5vdHg7gaJpZM4cVgrx>.