Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix integer division after migration to python 3 #34712

Merged
merged 1 commit into from
Aug 3, 2021
Merged

Fix integer division after migration to python 3 #34712

merged 1 commit into from
Aug 3, 2021

Conversation

fwyzard
Copy link
Contributor

@fwyzard fwyzard commented Jul 31, 2021

PR description:

The migration to python 3 changed the rules for integer divisions to return floating point numbers instead of truncating to the lower integer.
Under some conditions, this change breaks makeTrackValidation.py, which ultimately fails with the error:

  File "/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-27-2300/python/Validation/RecoTrack/plotting/plotting.py", line 457, in _getYminMaxAroundMedian
    return (yvals[ind_min], yvals[ind_max])
TypeError: list indices must be integers or slices, not float

This PR changes the division used to compute the bins to the "integer division" operator //, recovering the intended behaviour.

PR validation:

With these changes, makeTrackValidation.py works as intended on the same DQM files where it was failing before.

PR status:

This PR is a backport of #34711 to CMSSW_12_0_X

@fwyzard
Copy link
Contributor Author

fwyzard commented Jul 31, 2021

type bugfix

@fwyzard
Copy link
Contributor Author

fwyzard commented Jul 31, 2021

backport #34711

@fwyzard
Copy link
Contributor Author

fwyzard commented Jul 31, 2021

please test

@cmsbuild cmsbuild added this to the CMSSW_12_0_X milestone Jul 31, 2021
@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 31, 2021 •
edited
Loading

A new Pull Request was created by @fwyzard (Andrea Bocci) for CMSSW_12_0_X.

It involves the following packages:

  • Validation/RecoTrack (dqm)

@andrius-k, @kmaeshima, @ErnestaP, @ahmad3213, @jfernan2, @rvenditti can you please review it and eventually sign? Thanks.
@felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @ebrondol, @wmtford, @mmusich, @mtosi, @dgulhan this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy, @perrotta you are the release manager for this.

cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a694c4/17382/summary.html
COMMIT: 6ce230b
CMSSW: CMSSW_12_0_X_2021-07-30-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34712/17382/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

  • 11603.011603.0_SingleElectronPt1000+2021+SingleElectronPt1000_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_SingleElectronPt1000+2021+SingleElectronPt1000_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log
  • 11634.011634.0_TTbar_14TeV+2021+TTbar_14TeV_TuneCP5_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_TTbar_14TeV+2021+TTbar_14TeV_TuneCP5_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log
  • 11601.011601.0_SingleElectronPt10+2021+SingleElectronPt10_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_SingleElectronPt10+2021+SingleElectronPt10_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log
Expand to see more relval errors ...

Comparison Summary

The workflows 140.53 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 1303 differences found in the comparisons
  • DQMHistoTests: Total files compared: 39
  • DQMHistoTests: Total histograms compared: 2998564
  • DQMHistoTests: Total failures: 3678
  • DQMHistoTests: Total nulls: 19
  • DQMHistoTests: Total successes: 2994845
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 45.703 KiB( 38 files compared)
  • DQMHistoSizes: changed ( 140.53 ): 44.531 KiB Hcal/DigiRunHarvesting
  • DQMHistoSizes: changed ( 140.53 ): 1.172 KiB RPC/DCSInfo
  • Checked 165 log files, 37 edm output root files, 39 DQM output files
  • TriggerResults: no differences found

@perrotta
Copy link
Contributor

perrotta commented Aug 2, 2021

backport of #34708
(The backport is indeed needed, because #34708 is only merged for 12_1_X)

@perrotta
Copy link
Contributor

perrotta commented Aug 2, 2021

please test
(Trying to refresh the flag, even if I won't really care about those "RelVal input" failures, if they show up again)

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 2, 2021

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a694c4/17409/summary.html
COMMIT: 6ce230b
CMSSW: CMSSW_12_0_X_2021-08-01-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34712/17409/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

  • 11634.2411634.24_TTbar_14TeV+2021_0T+TTbar_14TeV_TuneCP5_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_TTbar_14TeV+2021_0T+TTbar_14TeV_TuneCP5_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log
  • 11601.011601.0_SingleElectronPt10+2021+SingleElectronPt10_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_SingleElectronPt10+2021+SingleElectronPt10_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log
  • 11602.011602.0_SingleElectronPt35+2021+SingleElectronPt35_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_SingleElectronPt35+2021+SingleElectronPt35_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log
Expand to see more relval errors ...

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 2 differences found in the comparisons
  • DQMHistoTests: Total files compared: 39
  • DQMHistoTests: Total histograms compared: 2998564
  • DQMHistoTests: Total failures: 7
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2998535
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
  • Checked 165 log files, 37 edm output root files, 39 DQM output files
  • TriggerResults: no differences found

@perrotta
Copy link
Contributor

perrotta commented Aug 3, 2021

@jfernan2 this is basically identical to your #34708, now merged in the master

@jfernan2
Copy link
Contributor

jfernan2 commented Aug 3, 2021

+1

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 3, 2021

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_0_X IBs (but tests are reportedly failing) and once validation in the development release cycle CMSSW_12_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy, @perrotta (and backports should be raised in the release meeting by the corresponding L2)

@perrotta
Copy link
Contributor

perrotta commented Aug 3, 2021

+1

@perrotta
Copy link
Contributor

perrotta commented Aug 3, 2021

merge

@cmsbuild cmsbuild merged commit 0cb28ab into cms-sw:CMSSW_12_0_X Aug 3, 2021
@cmsbuild cmsbuild mentioned this pull request Aug 3, 2021
Merged
@fwyzard fwyzard deleted the fix_Validation_RecoTrack_for_python3 branch August 18, 2021 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
backport-ok bug-fix dqm-approved fully-signed orp-approved tests-rejected
Projects
None yet
Milestone
CMSSW_12_0_X
Development

Successfully merging this pull request may close these issues.
4 participants
@fwyzard @cmsbuild @perrotta @jfernan2