Update Millepede to V04-11-01 and switch to git #7660

iarspider · 2022-03-02T14:40:40Z

No description provided.

iarspider · 2022-03-02T14:40:48Z

please test

cmsbuild · 2022-03-02T14:41:18Z

A new Pull Request was created by @iarspider for branch IB/CMSSW_12_3_X/master.

@smuzaffar, @iarspider can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @qliphy you are the release manager for this.
cms-bot commands are listed here

cmsbuild · 2022-03-02T19:49:48Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8775e5/22772/summary.html
COMMIT: ab22edf
CMSSW: CMSSW_12_3_X_2022-03-01-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/7660/22772/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8775e5/22772/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8775e5/22772/git-merge-result

Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 24 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3987333
DQMHistoTests: Total failures: 859
DQMHistoTests: Total nulls: 56
DQMHistoTests: Total successes: 3986396
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -49771.308 KiB( 48 files compared)
DQMHistoSizes: changed ( 11634.0,... ): -7205.463 KiB HLT/Tracking
DQMHistoSizes: changed ( 11634.0,... ): -1091.170 KiB HLT/BPH
DQMHistoSizes: changed ( 11634.0,... ): 1.415 KiB HLT/Filters
Checked 204 log files, 45 edm output root files, 49 DQM output files
TriggerResults: found differences in 7 / 48 workflows

smuzaffar · 2022-03-03T07:47:16Z

assign alca

Do we still use millepede in cmssw? I see that this external only provides one executable pede (and no libs and no header files) [a] and there is no ref to execution of pede in any unit tests or relvals? If it is still in use then can we add at least one unit test for this?

[a]

> ls /cvmfs/cms-ci.cern.ch/week0/PR_2044ecd5/slc7_amd64_gcc10/external/millepede/V04-11-01-3146f936e8445dced8ee050d2ed81a0a/
bin  etc
> ls /cvmfs/cms-ci.cern.ch/week0/PR_2044ecd5/slc7_amd64_gcc10/external/millepede/V04-11-01-3146f936e8445dced8ee050d2ed81a0a/bin/
pede

cmsbuild · 2022-03-03T07:47:31Z

New categories assigned: alca

@yuanchao,@francescobrivio,@malbouis,@tvami you have been requested to review this Pull request/Issue and eventually sign? Thanks

tvami · 2022-03-03T14:03:06Z

Hi @smuzaffar , yes MillePede is still used by the tracker alignment group! Let me tag @cms-sw/trk-dpg-l2 for further comments

mmusich · 2022-03-03T14:05:57Z

and there is no ref to execution of pede in any unit tests or relvals?

that's not correct. There's minimal testing in the relval workflow 1001.0.
Looking at the log of one recent IB, though, it looks like it's silently failing: log

smuzaffar · 2022-03-04T16:10:19Z

@mmusich , my bad, yes it is used by wf 1001.0 . @tvami any objections on merging this?

smuzaffar · 2022-03-04T16:11:20Z

@cms-sw/alca-l2 , can you also please add a unit test for testing pede?

smuzaffar · 2022-03-04T16:13:45Z

I will add a simple unit test which just runs pede -t to make it runs

mmusich · 2022-03-04T17:35:36Z

I will add a simple unit test which just runs pede -t to make it runs

thanks. In a slightly longer timescale one could think of putting together also a more realistic test which runs pede as in a small alignment campaign setup, if @consuegs and @antoniovagnerini provide the configuration and the input binary files.

smuzaffar · 2022-03-04T18:26:09Z

please test with cms-sw/cmssw#37145

cmsbuild · 2022-03-04T23:10:44Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8775e5/22853/summary.html
COMMIT: ab22edf
CMSSW: CMSSW_12_3_X_2022-03-04-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/7660/22853/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8775e5/22853/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8775e5/22853/git-merge-result

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 35 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3987741
DQMHistoTests: Total failures: 16
DQMHistoTests: Total nulls: 4
DQMHistoTests: Total successes: 3987699
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 204 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

tvami · 2022-03-04T23:28:06Z

@smuzaffar no objections, and thanks for the unit test.

I also support making a real test with a small alignment campaign setup as Marco suggested

mmusich · 2022-03-11T14:27:56Z

millepede.spec

-
-Source: svn://svnsrv.desy.de/public/MillepedeII/tags/%{realversion}/?scheme=http&module=%{realversion}&output=/%{n}-%{realversion}.tgz
+### RPM external millepede V04-11-01
+%define tag 9ee817fc61fe3e1b6543a8a16f7bcd8e1f8c331f


@iarspider @smuzaffar for my own education from where does this tag come from?
If I look at: https://gitlab.desy.de/claus.kleinwort/millepede-ii/-/tags the tag corresponding to version V04-11-01 is rather 4f288098f84f274f5a241f921035af77bc07d5c8?
Can you explain?
Thanks
Cc: @consuegs

Nice catch! I will open a follow-up PR now

@mmusich , you are right, the commit does not match the tag. @iarspider cn you please fix this

@iarspider , do you remember how you ended up using 9ee817fc61fe3e1b6543a8a16f7bcd8e1f8c331f ?

I think I was just browsing history trying to check a few things, and copy-pasted wrong commit.

Update Millipede to V04-11-01 and switch to git

ab22edf

cmsbuild added externals-pending orp-pending pending-signatures tests-started labels Mar 2, 2022

cmsbuild added tests-approved and removed tests-started labels Mar 2, 2022

iarspider requested a review from smuzaffar March 2, 2022 22:30

iarspider assigned smuzaffar Mar 2, 2022

cmsbuild added the alca-pending label Mar 3, 2022

cmsbuild added requires-external tests-started and removed tests-approved labels Mar 4, 2022

cmsbuild added tests-approved and removed tests-started labels Mar 4, 2022

smuzaffar approved these changes Mar 5, 2022

View reviewed changes

smuzaffar merged commit 8fd372c into IB/CMSSW_12_3_X/master Mar 5, 2022

smuzaffar deleted the update-millipede branch March 5, 2022 09:43

This was referenced Mar 5, 2022

Update pacparser to 1.3.7 #7663

Merged

[ROOT626] Updated root to tip of branch v6-26-00-patches #7670

Merged

[ROOT6] Updated root to tip of branch master #7671

Merged

sys/sysctl has been removed from newer glibc-header cms-externals/classlib#2

Merged

This was referenced Mar 11, 2022

introduce alignment campaign-like unit test for pede cms-sw/cmssw#37211

Merged

[12.2.X] Introduce alignment campaign-like unit test for pede cms-sw/cmssw#37212

Merged

mmusich reviewed Mar 11, 2022

View reviewed changes

iarspider mentioned this pull request Mar 11, 2022

Actually update Millipede to V04-11-01 #7685

Merged

mmusich mentioned this pull request Mar 13, 2022

[12.3.X] Introduce alignment campaign-like unit test for pede cms-sw/cmssw#37226

Merged

smuzaffar mentioned this pull request Mar 14, 2022

[12.3.X] Millipede to V04-11-01 #7687

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Millepede to V04-11-01 and switch to git #7660

Update Millepede to V04-11-01 and switch to git #7660

iarspider commented Mar 2, 2022

iarspider commented Mar 2, 2022

cmsbuild commented Mar 2, 2022

cmsbuild commented Mar 2, 2022

smuzaffar commented Mar 3, 2022

cmsbuild commented Mar 3, 2022

tvami commented Mar 3, 2022

mmusich commented Mar 3, 2022 •

edited

smuzaffar commented Mar 4, 2022

smuzaffar commented Mar 4, 2022

smuzaffar commented Mar 4, 2022

mmusich commented Mar 4, 2022

smuzaffar commented Mar 4, 2022

cmsbuild commented Mar 4, 2022

tvami commented Mar 4, 2022

mmusich Mar 11, 2022

iarspider Mar 11, 2022 •

edited

smuzaffar Mar 11, 2022

smuzaffar Mar 11, 2022

iarspider Mar 11, 2022

Update Millepede to V04-11-01 and switch to git #7660

Update Millepede to V04-11-01 and switch to git #7660

Conversation

iarspider commented Mar 2, 2022

iarspider commented Mar 2, 2022

cmsbuild commented Mar 2, 2022

cmsbuild commented Mar 2, 2022

Comparison Summary

smuzaffar commented Mar 3, 2022

cmsbuild commented Mar 3, 2022

tvami commented Mar 3, 2022

mmusich commented Mar 3, 2022 • edited

smuzaffar commented Mar 4, 2022

smuzaffar commented Mar 4, 2022

smuzaffar commented Mar 4, 2022

mmusich commented Mar 4, 2022

smuzaffar commented Mar 4, 2022

cmsbuild commented Mar 4, 2022

Comparison Summary

tvami commented Mar 4, 2022

mmusich Mar 11, 2022

Choose a reason for hiding this comment

iarspider Mar 11, 2022 • edited

Choose a reason for hiding this comment

smuzaffar Mar 11, 2022

Choose a reason for hiding this comment

smuzaffar Mar 11, 2022

Choose a reason for hiding this comment

iarspider Mar 11, 2022

Choose a reason for hiding this comment

mmusich commented Mar 3, 2022 •

edited

iarspider Mar 11, 2022 •

edited