[93X] Streamline Alignment PV Validation tool #19839

mmusich · 2017-07-20T16:42:34Z

Greetings,
this PR is meant to streamline and consolidate the PV tool:

possibly common methods are moved into an helper namespace;
make booking and filling of histograms uniform;
reduce duplications in the creation of the track variable bins (hardwiring all instances to a single std::map depending on the impact parameter type and track variables);
added profiles of the bias and resolution towards the BPix L1 ladder and module number (where useful in the startup pixel commissioning phase): this is supported also for the Phase-0 detector.
update the plotting macro according to the other changes.

cmsbuild · 2017-07-20T16:42:51Z

A new Pull Request was created by @mmusich (Marco Musich) for master.

It involves the following packages:

Alignment/OfflineValidation

@ghellwig, @arunhep, @cerminar, @cmsbuild, @franzoni, @davidlange6, @lpernie can you please review it and eventually sign? Thanks.
@mschrode, @ghellwig, @mmusich, @tocheng, @tlampen this is something you requested to watch as well.
@davidlange6 you are the release manager for this.

cms-bot commands are listed here

davidlange6 · 2017-07-20T18:42:49Z

please test

cmsbuild · 2017-07-20T18:43:06Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/21644/console Started: 2017/07/20 20:43

cmsbuild · 2017-07-20T21:02:41Z

+1
Tested at: 35dd5e9
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19839/21644/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
97d9ccd
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19839/21644/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19839/21644/git-merge-result

cmsbuild · 2017-07-20T21:02:44Z

Comparison job queued.

cmsbuild · 2017-07-20T23:07:09Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19839/21644/summary.html

Comparison Summary:

You potentially added 5 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 23
DQMHistoTests: Total histograms compared: 2346597
DQMHistoTests: Total failures: 232
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2346199
DQMHistoTests: Total skipped: 166
DQMHistoTests: Total Missing objects: 0
Checked 93 log files, 14 edm output root files, 23 DQM output files

franzoni · 2017-07-25T14:18:09Z

+1

cmsbuild · 2017-07-25T14:18:25Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @smuzaffar

davidlange6 · 2017-08-01T07:49:43Z

hi @mmusich - given the DQM dependency on Alignment/OfflineValidation, this package should not depend on rootgraphics. which code brings in this requirement?

mmusich · 2017-08-01T07:53:16Z

Hi @davidlange6, moving the definition of some helper methods to Alignment/OfflineValidation/interface/PVValidationHelpers.h required rootgraphics in the BuildFile to be able to compile.

davidlange6 · 2017-08-01T07:57:45Z

there is nothing I see in PVValidationHelpers that needs root graphics. Can you be more specific? these should be part of rootcore.

#include "TMath.h"
#include "TH1.h"
#include "TF1.h"

mmusich · 2017-08-01T08:12:17Z

@davidlange6 changingrootgraphics into rootcore I am getting:

tmp/slc6_amd64_gcc630/src/Alignment/OfflineValidation/src/AlignmentOfflineValidation/PVValidationHelpers.o: In function `PVValHelper::add(std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, TH1*, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TH1*> > >&, TH1*)':
PVValidationHelpers.cc:(.text+0x2f70): undefined reference to `TH1::StatOverflows(bool)'
tmp/slc6_amd64_gcc630/src/Alignment/OfflineValidation/src/AlignmentOfflineValidation/PVValidationHelpers.o: In function `PVValHelper::getMAD(TH1F*)':
PVValidationHelpers.cc:(.text+0x45a5): undefined reference to `TH1F::TH1F(char const*, char const*, int, double, double)'
tmp/slc6_amd64_gcc630/src/Alignment/OfflineValidation/src/AlignmentOfflineValidation/PVValidationHelpers.o: In function `PVValHelper::fitResiduals(TH1*)':
PVValidationHelpers.cc:(.text+0x47a7): undefined reference to `TF1::TF1(char const*, char const*, double, double, TF1::EAddToList)'
PVValidationHelpers.cc:(.text+0x47dc): undefined reference to `TFitResultPtr::~TFitResultPtr()'
PVValidationHelpers.cc:(.text+0x4800): undefined reference to `TFormula::GetParameter(int) const'
PVValidationHelpers.cc:(.text+0x4828): undefined reference to `TFormula::GetParameter(int) const'
PVValidationHelpers.cc:(.text+0x4850): undefined reference to `TF1::SetRange(double, double)'
PVValidationHelpers.cc:(.text+0x488a): undefined reference to `TFitResultPtr::~TFitResultPtr()'
PVValidationHelpers.cc:(.text+0x48ec): undefined reference to `TFormula::GetParameter(int) const'
PVValidationHelpers.cc:(.text+0x4914): undefined reference to `TFormula::GetParameter(int) const'
PVValidationHelpers.cc:(.text+0x492f): undefined reference to `TF1::GetParError(int) const'
PVValidationHelpers.cc:(.text+0x494a): undefined reference to `TF1::GetParError(int) const'
PVValidationHelpers.cc:(.text+0x4996): undefined reference to `TF1::~TF1()'
PVValidationHelpers.cc:(.text+0x4a8b): undefined reference to `TF1::~TF1()'
collect2: error: ld returned 1 exit status
config/SCRAM/GMake/Makefile.rules:2040: recipe for target 'tmp/slc6_amd64_gcc630/src/Alignment/OfflineValidation/src/AlignmentOfflineValidation/libAlignmentOfflineValidation.so' failed
gmake: *** [tmp/slc6_amd64_gcc630/src/Alignment/OfflineValidation/src/AlignmentOfflineValidation/libAlignmentOfflineValidation.so] Error 1
gmake: *** Waiting for unfinished jobs....
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

davidlange6 · 2017-08-01T08:19:30Z

seems I misread - should be roothistmatrix

…

On Aug 1, 2017, at 10:12 AM, Marco Musich ***@***.***> wrote: @davidlange6 changingrootgraphics than rootcore I am getting: tmp/slc6_amd64_gcc630/src/Alignment/OfflineValidation/src/AlignmentOfflineValidation/PVValidationHelpers.o: In function `PVValHelper::add(std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, TH1*, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TH1*> > >&, TH1*)': PVValidationHelpers.cc:(.text+0x2f70): undefined reference to `TH1::StatOverflows(bool)' tmp/slc6_amd64_gcc630/src/Alignment/OfflineValidation/src/AlignmentOfflineValidation/PVValidationHelpers.o: In function `PVValHelper::getMAD(TH1F*)': PVValidationHelpers.cc:(.text+0x45a5): undefined reference to `TH1F::TH1F(char const*, char const*, int, double, double)' tmp/slc6_amd64_gcc630/src/Alignment/OfflineValidation/src/AlignmentOfflineValidation/PVValidationHelpers.o: In function `PVValHelper::fitResiduals(TH1*)': PVValidationHelpers.cc:(.text+0x47a7): undefined reference to `TF1::TF1(char const*, char const*, double, double, TF1::EAddToList)' PVValidationHelpers.cc:(.text+0x47dc): undefined reference to `TFitResultPtr::~TFitResultPtr()' PVValidationHelpers.cc:(.text+0x4800): undefined reference to `TFormula::GetParameter(int) const' PVValidationHelpers.cc:(.text+0x4828): undefined reference to `TFormula::GetParameter(int) const' PVValidationHelpers.cc:(.text+0x4850): undefined reference to `TF1::SetRange(double, double)' PVValidationHelpers.cc:(.text+0x488a): undefined reference to `TFitResultPtr::~TFitResultPtr()' PVValidationHelpers.cc:(.text+0x48ec): undefined reference to `TFormula::GetParameter(int) const' PVValidationHelpers.cc:(.text+0x4914): undefined reference to `TFormula::GetParameter(int) const' PVValidationHelpers.cc:(.text+0x492f): undefined reference to `TF1::GetParError(int) const' PVValidationHelpers.cc:(.text+0x494a): undefined reference to `TF1::GetParError(int) const' PVValidationHelpers.cc:(.text+0x4996): undefined reference to `TF1::~TF1()' PVValidationHelpers.cc:(.text+0x4a8b): undefined reference to `TF1::~TF1()' collect2: error: ld returned 1 exit status config/SCRAM/GMake/Makefile.rules:2040: recipe for target 'tmp/slc6_amd64_gcc630/src/Alignment/OfflineValidation/src/AlignmentOfflineValidation/libAlignmentOfflineValidation.so' failed gmake: *** [tmp/slc6_amd64_gcc630/src/Alignment/OfflineValidation/src/AlignmentOfflineValidation/libAlignmentOfflineValidation.so] Error 1 gmake: *** Waiting for unfinished jobs.... gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

cmsbuild · 2017-08-08T15:20:44Z

Pull request #19839 was updated. @ghellwig, @arunhep, @cerminar, @cmsbuild, @franzoni, @lpernie can you please check and sign again.

…n in case of merged output files

cmsbuild · 2017-08-08T15:26:49Z

Pull request #19839 was updated. @ghellwig, @arunhep, @cerminar, @cmsbuild, @franzoni, @lpernie can you please check and sign again.

mmusich · 2017-08-08T15:28:28Z

@davidlange6 @arunhep
sorry there was an issue in the unit tests due to the unprotected calculation of the median in case of empty histograms leading to the error in:

----- Begin Fatal Exception 08-Aug-2017 13:30:27 CEST-----------------------
An exception of category 'FatalRootError' occurred while
   [0] Calling endJob for module PrimaryVertexValidation/'PVValidation'
   Additional Info:
      [a] Fatal Root Error: @SUB=TH1F::ComputeIntegral
Integral = zero

----- End Fatal Exception -------------------------------------------------

now is fixed. I profited to add an additional commit that was left behind at the last push.
Now it's ready for tests. Thanks

arunhep · 2017-08-08T15:33:18Z

@cmsbuild please test

cmsbuild · 2017-08-08T15:33:36Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/22148/console Started: 2017/08/08 17:34

cmsbuild · 2017-08-08T17:24:34Z

+1
Tested at: cbb2414
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19839/22148/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
e4b262a
c925d3e
65ad419
eb460ac
947a871
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19839/22148/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19839/22148/git-merge-result

cmsbuild · 2017-08-08T17:24:36Z

Comparison job queued.

cmsbuild · 2017-08-08T18:55:05Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19839/22148/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 189 differences found in the comparisons
DQMHistoTests: Total files compared: 25
DQMHistoTests: Total histograms compared: 2646495
DQMHistoTests: Total failures: 1401
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2644912
DQMHistoTests: Total skipped: 181
DQMHistoTests: Total Missing objects: 0
Checked 102 log files, 14 edm output root files, 25 DQM output files

arunhep · 2017-08-08T21:51:07Z

+1

cmsbuild · 2017-08-08T21:51:27Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @smuzaffar (and backports should be raised in the release meeting by the corresponding L2)

davidlange6 · 2017-08-09T07:05:23Z

+1

cmsbuild added this to the CMSSW_9_3_X milestone Jul 20, 2017

cmsbuild added alca-pending comparison-pending orp-pending pending-signatures tests-pending labels Jul 20, 2017

cmsbuild added tests-started and removed tests-pending labels Jul 20, 2017

cmsbuild added tests-approved and removed tests-started labels Jul 20, 2017

cmsbuild added comparison-available and removed comparison-pending labels Jul 20, 2017

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Jul 25, 2017

cmsbuild removed the alca-approved label Aug 1, 2017

allow correct fetching of extrema in PV Validation plotting macro eve…

cbb2414

…n in case of merged output files

cmsbuild added tests-started and removed tests-pending labels Aug 8, 2017

cmsbuild added tests-approved and removed tests-started labels Aug 8, 2017

cmsbuild added comparison-available and removed comparison-pending labels Aug 8, 2017

cmsbuild removed the alca-pending label Aug 8, 2017

cmsbuild added alca-approved fully-signed and removed pending-signatures labels Aug 8, 2017

cmsbuild added orp-approved and removed orp-pending labels Aug 9, 2017

cmsbuild merged commit b6fcc36 into cms-sw:master Aug 9, 2017

mmusich mentioned this pull request Aug 9, 2017

[92X] Cleanup and Streamline Alignment PV Validation tool #20087

Merged

mmusich added a commit to mmusich/cmssw that referenced this pull request Sep 11, 2017

Change reference file as consequence of cms-sw#19839

b75a169

mmusich added a commit to mmusich/cmssw that referenced this pull request Oct 27, 2017

Change reference file as consequence of cms-sw#19839

44377ea

mmusich added a commit to mmusich/cmssw that referenced this pull request Oct 30, 2017

Change reference file as consequence of cms-sw#19839

8ef3354

mmusich added a commit to mmusich/cmssw that referenced this pull request Nov 8, 2017

Change reference file as consequence of cms-sw#19839

2e1a2c1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[93X] Streamline Alignment PV Validation tool #19839

[93X] Streamline Alignment PV Validation tool #19839

mmusich commented Jul 20, 2017 •

edited

cmsbuild commented Jul 20, 2017

davidlange6 commented Jul 20, 2017

cmsbuild commented Jul 20, 2017 •

edited

cmsbuild commented Jul 20, 2017

cmsbuild commented Jul 20, 2017

cmsbuild commented Jul 20, 2017

franzoni commented Jul 25, 2017

cmsbuild commented Jul 25, 2017

davidlange6 commented Aug 1, 2017

mmusich commented Aug 1, 2017

davidlange6 commented Aug 1, 2017

mmusich commented Aug 1, 2017 •

edited

davidlange6 commented Aug 1, 2017 via email

cmsbuild commented Aug 8, 2017

cmsbuild commented Aug 8, 2017

mmusich commented Aug 8, 2017

arunhep commented Aug 8, 2017

cmsbuild commented Aug 8, 2017 •

edited

cmsbuild commented Aug 8, 2017

cmsbuild commented Aug 8, 2017

cmsbuild commented Aug 8, 2017

arunhep commented Aug 8, 2017

cmsbuild commented Aug 8, 2017

davidlange6 commented Aug 9, 2017

[93X] Streamline Alignment PV Validation tool #19839

[93X] Streamline Alignment PV Validation tool #19839

Conversation

mmusich commented Jul 20, 2017 • edited

cmsbuild commented Jul 20, 2017

davidlange6 commented Jul 20, 2017

cmsbuild commented Jul 20, 2017 • edited

cmsbuild commented Jul 20, 2017

cmsbuild commented Jul 20, 2017

cmsbuild commented Jul 20, 2017

franzoni commented Jul 25, 2017

cmsbuild commented Jul 25, 2017

davidlange6 commented Aug 1, 2017

mmusich commented Aug 1, 2017

davidlange6 commented Aug 1, 2017

mmusich commented Aug 1, 2017 • edited

davidlange6 commented Aug 1, 2017 via email

cmsbuild commented Aug 8, 2017

cmsbuild commented Aug 8, 2017

mmusich commented Aug 8, 2017

arunhep commented Aug 8, 2017

cmsbuild commented Aug 8, 2017 • edited

cmsbuild commented Aug 8, 2017

cmsbuild commented Aug 8, 2017

cmsbuild commented Aug 8, 2017

arunhep commented Aug 8, 2017

cmsbuild commented Aug 8, 2017

davidlange6 commented Aug 9, 2017

mmusich commented Jul 20, 2017 •

edited

cmsbuild commented Jul 20, 2017 •

edited

mmusich commented Aug 1, 2017 •

edited

cmsbuild commented Aug 8, 2017 •

edited