introduce alignment campaign-like unit test for `pede` #37211

mmusich · 2022-03-11T13:48:37Z

PR description:

As discussed here cms-sw/cmsdist#7660 (comment) introduce a pede execution based off a real (small-scale) MC alignment campaign (cosmics-only). The needed input files to steer the execution are at: /store/group/alca_global/tkal_millepede_tests/

PR validation:

run scram b runtests_testPedeCampaign, ~~but at the moment (for reason that are being investigated) the pede execution gets stuck (starting from CMSSW_12_3_0_pre3 onwards), while it run fine in previous pre-releases~~ (The issue, due to a wrong commit for the pede spec file in cmsdist has been solved in cms-sw/cmsdist#7685)

if this PR is a backport please specify the original PR and why you need to backport that PR:

A backport exists, to show the infrastructure is working.

mmusich · 2022-03-11T13:49:16Z

@consuegs @antoniovagnerini @tsusa FYI

cmsbuild · 2022-03-11T13:55:58Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37211/28813

This PR adds an extra 12KB to repository

cmsbuild · 2022-03-11T13:56:18Z

A new Pull Request was created by @mmusich (Marco Musich) for master.

It involves the following packages:

Alignment/MillePedeAlignmentAlgorithm (alca)

@cmsbuild, @malbouis, @tvami, @yuanchao, @francescobrivio can you please review it and eventually sign? Thanks.
@mmusich, @adewit, @tocheng, @tlampen this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

mmusich · 2022-03-11T13:59:30Z

please test

cmsbuild · 2022-03-11T18:54:47Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a8f699/23048/summary.html
COMMIT: 4055172
CMSSW: CMSSW_12_3_X_2022-03-11-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37211/23048/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test testPedeCampaign had ERRORS

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3695161
DQMHistoTests: Total failures: 8
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3695131
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 204 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

mmusich · 2022-03-11T18:55:40Z

test parameters:

pull_request = Actually update Millipede to V04-11-01 cmsdist#7685

mmusich · 2022-03-11T18:55:52Z

@cmsbuild, please test

cmsbuild · 2022-03-11T23:22:32Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a8f699/23053/summary.html
COMMIT: 4055172
CMSSW: CMSSW_12_4_X_2022-03-11-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37211/23053/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a8f699/23053/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a8f699/23053/git-merge-result

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3695153
DQMHistoTests: Total failures: 50
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3695081
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 204 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

tvami · 2022-03-13T18:30:18Z

Since cms-sw/cmsdist#7685 is merged, should the status of this be changed to "ready for review"?

cmsbuild · 2022-03-13T19:33:44Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37211/28826

This PR adds an extra 12KB to repository

cmsbuild · 2022-03-13T19:34:06Z

Pull request #37211 was updated. @cmsbuild, @malbouis, @tvami, @yuanchao, @francescobrivio can you please check and sign again.

mmusich · 2022-03-13T19:34:16Z

@cmsbuild, please test

cmsbuild · 2022-03-13T23:33:25Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a8f699/23077/summary.html
COMMIT: d0c2a12
CMSSW: CMSSW_12_4_X_2022-03-13-0000/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37211/23077/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3695161
DQMHistoTests: Total failures: 14
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3695125
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 204 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

mmusich · 2022-03-14T16:13:42Z

@cms-sw/alca-l2 any further thoughts?

tvami · 2022-03-14T16:16:27Z

+alca

new unit test passed

cmsbuild · 2022-03-14T16:16:49Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-03-14T16:49:52Z

+1

Unit test succeeds

qliphy · 2022-03-15T07:31:59Z

@mmusich There seems to be a related IB unit issue after merging this PR. Would you please have a look?

https://cmssdt.cern.ch/SDT/cgi-bin/logreader/slc7_amd64_gcc10/CMSSW_12_4_X_2022-03-14-2300/unitTestLogs/Alignment/TrackerAlignment#/292-292

Processing allMillePede.C...

---> test test_PixelBaryCentreTool had ERRORS
TestTime:3600
^^^^ End Test test_PixelBaryCentreTool ^^^^

Tests for package Alignment/TrackerAlignment ran.

mmusich · 2022-03-15T08:32:38Z

@qliphy

There seems to be a related IB unit issue after merging this PR. Would you please have a look?

No. This PR has been merged in CMSSW_12_4_X_2022-03-14-2300 according to https://cmssdt.cern.ch/SDT/html/cmssdt-ib/#/ib/CMSSW_12_4_X, and I see that this test has been failing since earlier (e.g. CMSSW_12_4_X_2022-03-14-1100)

https://cmssdt.cern.ch/SDT/cgi-bin/logreader/rhel8_amd64_gcc10/CMSSW_12_4_X_2022-03-14-1100/unitTestLogs/Alignment/TrackerAlignment#/6

therefore I conclude that the failure is unrelated.

mmusich · 2022-03-15T14:11:28Z

Actually it failed even earlier: https://cmssdt.cern.ch/SDT/cgi-bin/logreader/rhel8_amd64_gcc10/CMSSW_12_4_X_2022-03-12-1100/unitTestLogs/Alignment/TrackerAlignment#/3 (CMSSW_12_4_X_2022-03-12-1100) so it's clearly not related.

qliphy · 2022-03-15T14:20:41Z

Ok, thanks @mmusich The new IB [CMSSW_12_4_X_2022-03-15-1100] so far looks good. We will then merge the backported PRs later soon.

mmusich mentioned this pull request Mar 11, 2022

[12.2.X] Introduce alignment campaign-like unit test for pede #37212

Merged

cmsbuild added this to the CMSSW_12_4_X milestone Mar 11, 2022

cmsbuild added alca-pending code-checks-pending orp-pending pending-signatures tests-pending labels Mar 11, 2022

mmusich changed the title ~~introduce alignment campaing-like unit test for pede~~ introduce alignment campaign-like unit test for pede Mar 11, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 11, 2022

cmsbuild added tests-started and removed tests-pending labels Mar 11, 2022

cmsbuild added tests-rejected and removed tests-started labels Mar 11, 2022

cmsbuild added requires-external tests-started and removed tests-rejected labels Mar 11, 2022

cmsbuild added tests-approved and removed tests-started labels Mar 11, 2022

introduce real data unit test for pede

d0c2a12

mmusich force-pushed the addMillePedeTestingSetup branch from 4055172 to d0c2a12 Compare March 13, 2022 19:25

mmusich marked this pull request as ready for review March 13, 2022 19:30

cmsbuild removed the requires-external label Mar 13, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 13, 2022

cmsbuild added tests-started and removed tests-pending labels Mar 13, 2022

cmsbuild added tests-approved and removed tests-started labels Mar 13, 2022

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Mar 14, 2022

cmsbuild added orp-approved and removed orp-pending labels Mar 14, 2022

cmsbuild merged commit ce4f5b2 into cms-sw:master Mar 14, 2022

mmusich deleted the addMillePedeTestingSetup branch March 14, 2022 17:06

mmusich mentioned this pull request Mar 28, 2024

add a unit test for conversion of millepede.res files into DB payloads #44570

Merged

mmusich mentioned this pull request Apr 11, 2024

[14.0.X] add a unit test for conversion of millepede.res files into DB payloads #44705

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

introduce alignment campaign-like unit test for `pede` #37211

introduce alignment campaign-like unit test for `pede` #37211

mmusich commented Mar 11, 2022 •

edited

mmusich commented Mar 11, 2022

cmsbuild commented Mar 11, 2022

cmsbuild commented Mar 11, 2022

mmusich commented Mar 11, 2022

cmsbuild commented Mar 11, 2022

mmusich commented Mar 11, 2022

mmusich commented Mar 11, 2022

cmsbuild commented Mar 11, 2022

tvami commented Mar 13, 2022

cmsbuild commented Mar 13, 2022

cmsbuild commented Mar 13, 2022

mmusich commented Mar 13, 2022

cmsbuild commented Mar 13, 2022

mmusich commented Mar 14, 2022

tvami commented Mar 14, 2022

cmsbuild commented Mar 14, 2022

perrotta commented Mar 14, 2022

qliphy commented Mar 15, 2022

mmusich commented Mar 15, 2022 •

edited

mmusich commented Mar 15, 2022

qliphy commented Mar 15, 2022

introduce alignment campaign-like unit test for pede #37211

introduce alignment campaign-like unit test for pede #37211

Conversation

mmusich commented Mar 11, 2022 • edited

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

mmusich commented Mar 11, 2022

cmsbuild commented Mar 11, 2022

cmsbuild commented Mar 11, 2022

mmusich commented Mar 11, 2022

cmsbuild commented Mar 11, 2022

Unit Tests

Comparison Summary

mmusich commented Mar 11, 2022

mmusich commented Mar 11, 2022

cmsbuild commented Mar 11, 2022

Comparison Summary

tvami commented Mar 13, 2022

cmsbuild commented Mar 13, 2022

cmsbuild commented Mar 13, 2022

mmusich commented Mar 13, 2022

cmsbuild commented Mar 13, 2022

Comparison Summary

mmusich commented Mar 14, 2022

tvami commented Mar 14, 2022

cmsbuild commented Mar 14, 2022

perrotta commented Mar 14, 2022

qliphy commented Mar 15, 2022

mmusich commented Mar 15, 2022 • edited

mmusich commented Mar 15, 2022

qliphy commented Mar 15, 2022

introduce alignment campaign-like unit test for `pede` #37211

introduce alignment campaign-like unit test for `pede` #37211

mmusich commented Mar 11, 2022 •

edited

mmusich commented Mar 15, 2022 •

edited