Some more python 2to3 fixes for HLT #34583

Martin-Grunewald · 2021-07-22T04:49:32Z

PR description:

Some more python 2to3 fixes for HLT

PR validation:

TSG tests

if this PR is a backport please specify the original PR and why you need to backport that PR:

N/A

cmsbuild · 2021-07-22T05:23:09Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34583/24134

This PR adds an extra 16KB to repository

cmsbuild · 2021-07-22T05:23:36Z

A new Pull Request was created by @Martin-Grunewald (Martin Grunewald) for master.

It involves the following packages:

HLTrigger/Configuration (hlt)

@Martin-Grunewald, @cmsbuild can you please review it and eventually sign? Thanks.
@makortel this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy, @perrotta you are the release manager for this.

cms-bot commands are listed here

Martin-Grunewald · 2021-07-22T06:26:09Z

please test

Martin-Grunewald · 2021-07-22T06:26:23Z

urgent

Martin-Grunewald · 2021-07-22T06:26:35Z

type bug-fix

cmsbuild · 2021-07-22T08:33:58Z

-1

Failed Tests: RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3026dc/17076/summary.html
COMMIT: de068d0
CMSSW: CMSSW_12_0_X_2021-07-21-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34583/17076/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

11634.911

----- Begin Fatal Exception 22-Jul-2021 09:26:22 CEST-----------------------
An exception of category 'StdException' occurred while
   [0] Running EventSetup component DDDetectorESProducer/'
Exception Message:
A std::exception was thrown.
An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file Geometry/VeryForwardData/data/RP_Vertical_Device/2021/v1/RP_Vertical_Device.xml anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34583/17076/CMSSW_12_0_X_2021-07-21-2300/poison:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34583/17076/CMSSW_12_0_X_2021-07-21-2300/src:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34583/17076/CMSSW_12_0_X_2021-07-21-2300/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-21-2300/src:/cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-21-2300/external/slc7_amd64_gcc900/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/11634.911_TTbar_14TeV+2021_DD4hep+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA

dd4hep: Error interpreting XML nodes of type <Include/>
dd4hep: Error interpreting XML nodes of type <IncludeSection/>
dd4hep: while parsing /cvmfs/cms-ib.cern.ch/nweek-02690/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-21-2300/src/Geometry/CMSCommonData/data/dd4hep/cmsExtendedGeometry2021.xml
dd4hep: with plugin:DD4hep_CompactLoader
----- End Fatal Exception -------------------------------------------------

Martin-Grunewald · 2021-07-22T09:04:59Z

Failure unrelated to this PR, also in baseline IB CMSSW_12_0_X_2021-07-21-2300.

Martin-Grunewald · 2021-07-22T09:05:05Z

+1

cmsbuild · 2021-07-22T09:05:17Z

This pull request is fully signed and it will be integrated in one of the next master IBs (but tests are reportedly failing). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy, @perrotta (and backports should be raised in the release meeting by the corresponding L2)

Martin-Grunewald · 2021-07-22T10:31:22Z

please test with PR #34534

Martin-Grunewald · 2021-07-22T10:32:24Z

please test with #34534

cmsbuild · 2021-07-22T14:38:46Z

-1

Failed Tests: UnitTests RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3026dc/17083/summary.html
COMMIT: de068d0
CMSSW: CMSSW_12_0_X_2021-07-21-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34583/17083/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test test_MuonGeometryDBConverter had ERRORS
---> test testDD4hepCompactView had ERRORS
---> test DetectorDescriptionDDCMSTestDriver had ERRORS
---> test testDD4hepFilteredView had ERRORS
and more ...

RelVals

11634.911

----- Begin Fatal Exception 22-Jul-2021 15:40:26 CEST-----------------------
An exception of category 'StdException' occurred while
   [0] Running EventSetup component DDDetectorESProducer/'
Exception Message:
A std::exception was thrown.
An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file Geometry/VeryForwardData/data/RP_Vertical_Device/2021/v1/RP_Vertical_Device.xml anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34583/17083/CMSSW_12_0_X_2021-07-21-2300/poison:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34583/17083/CMSSW_12_0_X_2021-07-21-2300/src:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34583/17083/CMSSW_12_0_X_2021-07-21-2300/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-21-2300/src:/cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-21-2300/external/slc7_amd64_gcc900/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/11634.911_TTbar_14TeV+2021_DD4hep+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA

dd4hep: Error interpreting XML nodes of type <Include/>
dd4hep: Error interpreting XML nodes of type <IncludeSection/>
dd4hep: while parsing /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34583/17083/CMSSW_12_0_X_2021-07-21-2300/src/Geometry/CMSCommonData/data/dd4hep/cmsExtendedGeometry2021.xml
dd4hep: with plugin:DD4hep_CompactLoader
----- End Fatal Exception -------------------------------------------------

perrotta · 2021-07-22T20:17:35Z

please test
(The "clean" CMSSW_12_0_X_2021-07-22-1700 IB should be available as baseline for tests now)

cmsbuild · 2021-07-22T21:53:13Z

-1

Failed Tests: RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3026dc/17113/summary.html
COMMIT: de068d0
CMSSW: CMSSW_12_0_X_2021-07-22-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34583/17113/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

11634.911

----- Begin Fatal Exception 22-Jul-2021 22:47:33 CEST-----------------------
An exception of category 'StdException' occurred while
   [0] Running EventSetup component DDDetectorESProducer/'
Exception Message:
A std::exception was thrown.
An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file Geometry/VeryForwardData/data/RP_Vertical_Device/2021/v1/RP_Vertical_Device.xml anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34583/17113/CMSSW_12_0_X_2021-07-22-1100/poison:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34583/17113/CMSSW_12_0_X_2021-07-22-1100/src:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34583/17113/CMSSW_12_0_X_2021-07-22-1100/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-22-1100/src:/cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-22-1100/external/slc7_amd64_gcc900/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/11634.911_TTbar_14TeV+2021_DD4hep+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA

dd4hep: Error interpreting XML nodes of type <Include/>
dd4hep: Error interpreting XML nodes of type <IncludeSection/>
dd4hep: while parsing /cvmfs/cms-ib.cern.ch/nweek-02690/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-22-1100/src/Geometry/CMSCommonData/data/dd4hep/cmsExtendedGeometry2021.xml
dd4hep: with plugin:DD4hep_CompactLoader
----- End Fatal Exception -------------------------------------------------

qliphy · 2021-07-23T01:38:14Z

please test

cmsbuild · 2021-07-23T10:13:42Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3026dc/17126/summary.html
COMMIT: de068d0
CMSSW: CMSSW_12_0_X_2021-07-22-1700/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34583/17126/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

11601.011601.0_SingleElectronPt10+2021+SingleElectronPt10_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_SingleElectronPt10+2021+SingleElectronPt10_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log
11602.011602.0_SingleElectronPt35+2021+SingleElectronPt35_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_SingleElectronPt35+2021+SingleElectronPt35_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log
11603.011603.0_SingleElectronPt1000+2021+SingleElectronPt1000_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_SingleElectronPt1000+2021+SingleElectronPt1000_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log

Expand to see more relval errors ...

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 2998564
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2998541
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

Martin-Grunewald · 2021-07-23T10:57:24Z

There must be a problem with the PR test infrastructure - this PR could not possibly create such errors in the PR tests...

qliphy · 2021-07-23T11:01:47Z

It is time out issue
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3026dc/17126/runTheMatrixINPUT-results/11601.0_SingleElectronPt10+2021+SingleElectronPt10_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_SingleElectronPt10+2021+SingleElectronPt10_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log

also appeared in the IB: CMSSW_12_0_X_2021-07-22-1700

Let's re-trigger the test with new IB

qliphy · 2021-07-23T11:01:51Z

please test

cmsbuild · 2021-07-23T14:03:18Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3026dc/17140/summary.html
COMMIT: de068d0
CMSSW: CMSSW_12_0_X_2021-07-22-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34583/17140/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 2998564
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2998541
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

qliphy · 2021-07-23T16:50:36Z

+1

Some more python 2to3 fixes

de068d0

cmsbuild added this to the CMSSW_12_0_X milestone Jul 22, 2021

cmsbuild added code-checks-pending hlt-pending orp-pending pending-signatures tests-pending labels Jul 22, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 22, 2021

cmsbuild added tests-started urgent and removed tests-pending labels Jul 22, 2021

cmsbuild added the bug-fix label Jul 22, 2021

cmsbuild added tests-rejected and removed tests-started labels Jul 22, 2021

cmsbuild added fully-signed hlt-approved and removed hlt-pending pending-signatures labels Jul 22, 2021

cmsbuild added the tests-started label Jul 22, 2021

cmsbuild added tests-rejected and removed tests-started labels Jul 22, 2021

cmsbuild added tests-started and removed requires-external tests-rejected labels Jul 22, 2021

cmsbuild added tests-rejected and removed tests-started labels Jul 22, 2021

cmsbuild added tests-started and removed tests-rejected labels Jul 23, 2021

cmsbuild added tests-rejected and removed tests-started labels Jul 23, 2021

cmsbuild added tests-started and removed tests-rejected labels Jul 23, 2021

cmsbuild added tests-approved and removed tests-started labels Jul 23, 2021

cmsbuild added orp-approved and removed orp-pending labels Jul 23, 2021

cmsbuild merged commit fb848c1 into cms-sw:master Jul 23, 2021

Martin-Grunewald deleted the HLTpy2to3updates branch October 27, 2021 07:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Some more python 2to3 fixes for HLT #34583

Some more python 2to3 fixes for HLT #34583

Martin-Grunewald commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

Martin-Grunewald commented Jul 22, 2021

Martin-Grunewald commented Jul 22, 2021

Martin-Grunewald commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

Martin-Grunewald commented Jul 22, 2021

Martin-Grunewald commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

Martin-Grunewald commented Jul 22, 2021

Martin-Grunewald commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

perrotta commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

qliphy commented Jul 23, 2021

cmsbuild commented Jul 23, 2021

Martin-Grunewald commented Jul 23, 2021

qliphy commented Jul 23, 2021

qliphy commented Jul 23, 2021

cmsbuild commented Jul 23, 2021

qliphy commented Jul 23, 2021

Some more python 2to3 fixes for HLT #34583

Some more python 2to3 fixes for HLT #34583

Conversation

Martin-Grunewald commented Jul 22, 2021

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

Martin-Grunewald commented Jul 22, 2021

Martin-Grunewald commented Jul 22, 2021

Martin-Grunewald commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

RelVals

Martin-Grunewald commented Jul 22, 2021

Martin-Grunewald commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

Martin-Grunewald commented Jul 22, 2021

Martin-Grunewald commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

Unit Tests

RelVals

perrotta commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

RelVals

qliphy commented Jul 23, 2021

cmsbuild commented Jul 23, 2021

RelVals-INPUT

Comparison Summary

Martin-Grunewald commented Jul 23, 2021

qliphy commented Jul 23, 2021

qliphy commented Jul 23, 2021

cmsbuild commented Jul 23, 2021

Comparison Summary

qliphy commented Jul 23, 2021