Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Validation plots for APE and bug fixes #34828

Merged
merged 1 commit into from Aug 14, 2021
Merged

Validation plots for APE and bug fixes #34828

merged 1 commit into from Aug 14, 2021

Conversation

mteroerd
Copy link
Contributor

PR description:

Changes to the APE tool:

  • Bug fixes: Remove lines that were previously added to prevent memory leaks, but which created errors. Also tweak some lines in test/plottingTools.
  • Added a new plotting tool to allow for creation of plots with residual distributions plus some other validation plots.
  • Added an option to autoSubmitter, so that APE can be more easily measured on cosmics data sets.

PR validation:

Ran some basic tests of the code on the most recent APE campaign to confirm that the changes work as intended.

@mmusich
Copy link
Contributor

mmusich commented Aug 10, 2021

@cmsbuild, ping?

@cmsbuild cmsbuild added this to the CMSSW_12_1_X milestone Aug 10, 2021
@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34828/24608

  • Found files with invalid states:

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @mteroerd (Marius Teroerde) for master.

It involves the following packages:

  • Alignment/APEEstimation (alca)

@malbouis, @yuanchao, @cmsbuild, @tlampen, @pohsun, @francescobrivio, @tvami can you please review it and eventually sign? Thanks.
@mmusich, @tocheng, @adewit this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

@mmusich
Copy link
Contributor

mmusich commented Aug 10, 2021

@cmsbuild, please test

@cmsbuild
Copy link
Contributor

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d2cfd3/17672/summary.html
COMMIT: 41a504d
CMSSW: CMSSW_12_1_X_2021-08-10-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34828/17672/install.sh to create a dev area with all the needed externals and cmssw changes.

There was an issue with git-cms-merge-topic you can see the log here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d2cfd3/17672/git-merge-result

@mmusich
Copy link
Contributor

mmusich commented Aug 10, 2021

@smuzaffar how do I interpret the bizarre merge error above?

@smuzaffar
Copy link
Contributor

This is due to github service issue. Thing are getting back to normal now. I would suggest to restart the test

@qliphy
Copy link
Contributor

qliphy commented Aug 11, 2021

please test

@cmsbuild
Copy link
Contributor

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d2cfd3/17683/summary.html
COMMIT: 41a504d
CMSSW: CMSSW_12_1_X_2021-08-10-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34828/17683/install.sh to create a dev area with all the needed externals and cmssw changes.

There was an issue with git-cms-merge-topic you can see the log here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d2cfd3/17683/git-merge-result

@mteroerd
Bug fixes
eb9c7bd 
New plotting tool for validation plots

Option for cosmics tracks added

Remove wrong code
@cmsbuild
Copy link
Contributor

Pull request #34828 was updated. @malbouis, @yuanchao, @cmsbuild, @tlampen, @pohsun, @francescobrivio, @tvami can you please check and sign again.

@mmusich
Copy link
Contributor

mmusich commented Aug 11, 2021

@cmsbuild, please test

@cmsbuild
Copy link
Contributor

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d2cfd3/17694/summary.html
COMMIT: eb9c7bd
CMSSW: CMSSW_12_1_X_2021-08-10-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34828/17694/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

  • 23434.9923434.99_TTbar_14TeV+2026D49PU_PMXS1S2+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+HARVESTGlobalPU/step2_TTbar_14TeV+2026D49PU_PMXS1S2+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+HARVESTGlobalPU.log
  • 23434.2123434.21_TTbar_14TeV+2026D49PU_ProdLike+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTriggerPU+RecoGlobalPU+MiniAODPU/step2_TTbar_14TeV+2026D49PU_ProdLike+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTriggerPU+RecoGlobalPU+MiniAODPU.log
  • 23434.992123434.9921_TTbar_14TeV+2026D49PU_PMXS1S2ProdLike+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+MiniAODPU/step2_TTbar_14TeV+2026D49PU_PMXS1S2ProdLike+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+MiniAODPU.log
Expand to see more relval errors ...

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 8 differences found in the comparisons
  • DQMHistoTests: Total files compared: 39
  • DQMHistoTests: Total histograms compared: 2999420
  • DQMHistoTests: Total failures: 12
  • DQMHistoTests: Total nulls: 1
  • DQMHistoTests: Total successes: 2999385
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -0.004 KiB( 38 files compared)
  • DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
  • Checked 165 log files, 37 edm output root files, 39 DQM output files
  • TriggerResults: no differences found

@qliphy
Copy link
Contributor

qliphy commented Aug 13, 2021

please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d2cfd3/17753/summary.html
COMMIT: eb9c7bd
CMSSW: CMSSW_12_1_X_2021-08-12-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34828/17753/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 39
  • DQMHistoTests: Total histograms compared: 2999420
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2999397
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
  • Checked 165 log files, 37 edm output root files, 39 DQM output files
  • TriggerResults: no differences found

@tlampen
Copy link
Contributor

tlampen commented Aug 13, 2021

+alca

  • update to plotting tools, autoSubmitter for APE measurement, some bug fixes
  • no differences found in comparisons

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

@qliphy
Copy link
Contributor

qliphy commented Aug 14, 2021

+1

@cmsbuild cmsbuild merged commit 7b0f8cd into cms-sw:master Aug 14, 2021
@mteroerd mteroerd mentioned this pull request Aug 24, 2021
Merged
@qliphy
Copy link
Contributor

qliphy commented Aug 25, 2021

@mteroerd We may also need a backport to 12_0_X?

cmsbuild added a commit that referenced this pull request Aug 25, 2021
@cmsbuild
Merge pull request #34990 from mteroerd/ape_bugfix_backport
3776ea2 
Validation plots for APE and bug fixes: Backport of #34828
@mteroerd mteroerd mentioned this pull request Aug 25, 2021
Merged
cmsbuild added a commit that referenced this pull request Aug 26, 2021
@cmsbuild
Merge pull request #35007 from mteroerd/ape_bugfix_1200_backport
c3a3add 
Validation plots for APE and bug fixes: Backport of #34828
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
alca-approved code-checks-approved fully-signed orp-approved tests-approved
Projects
None yet
Milestone
CMSSW_12_1_X
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@mteroerd @mmusich @cmsbuild @smuzaffar @qliphy @tlampen