[Python3][Devel] testing python3 in devel IBs #28057

smuzaffar · 2019-09-24T08:30:11Z

Same as #28021 but for DEVEL IBs to test cms-sw/cmsdist#5241

cmsbuild · 2019-09-24T08:30:44Z

A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for CMSSW_11_0_DEVEL_X.

It involves the following packages:

Configuration/Applications
Configuration/PyReleaseValidation
FWCore/PythonUtilities

@smuzaffar, @Dr15Jones, @chayanit, @zhenhu, @kpedro88, @pgunnell, @franzoni, @cmsbuild, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @wddgit this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2019-09-24T08:33:55Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmsdist#5241
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2637/console Started: 2019/09/24 11:34
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2647/console Started: 2019/09/24 15:01

cmsbuild · 2019-09-24T11:27:01Z

-1

Tested at: 92d000b

Build:

I found compilation error when building:

+ DEPS='py2-PyYAML '
+ '[' 'py2-PyYAML ' '!=' '' ']'
+ echo 'ERROR: Missing dependencies for py2-hepdata-lib (python2) found: py2-PyYAML '
ERROR: Missing dependencies for py2-hepdata-lib (python2) found: py2-PyYAML 
+ exit 1
error: Bad exit status from /build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.E05MCY (%build)


RPM build errors:
Bad exit status from /build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.E05MCY (%build)

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-742fa0/2637/summary.html

cmsbuild · 2019-09-24T14:49:08Z

-1

Tested at: 92d000b

Build:

I found compilation error when building:

+ DEPS='py2-PyYAML '
+ '[' 'py2-PyYAML ' '!=' '' ']'
+ echo 'ERROR: Missing dependencies for py2-hepdata-lib (python2) found: py2-PyYAML '
ERROR: Missing dependencies for py2-hepdata-lib (python2) found: py2-PyYAML 
+ exit 1
error: Bad exit status from /build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.etmd3z (%build)


RPM build errors:
Bad exit status from /build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.etmd3z (%build)

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-742fa0/2647/summary.html

cmsbuild · 2019-09-24T15:16:38Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmsdist#5241
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2651/console Started: 2019/09/24 17:17

cmsbuild · 2019-09-24T20:03:09Z

-1

Tested at: 92d000b

Build:

I found compilation error when building:

File "/build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/slc7_amd64_gcc820/external/py2-pip/9.0.3-nmpfii/lib/python3.6/site-packages/pip/req/req_install.py", line 878, in install
spinner=spinner,
File "/build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/slc7_amd64_gcc820/external/py2-pip/9.0.3-nmpfii/lib/python3.6/site-packages/pip/utils/__init__.py", line 707, in call_subprocess
% (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command "/build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/slc7_amd64_gcc820/external/python3/3.6.4-pafccj/bin/python3 -u -c "import setuptools, tokenize;__file__='/build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/pip-build-tyq1z3bp/numpy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" --hdf5=/build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/slc7_amd64_gcc820/external/hdf5/1.8.17-pafccj --bzip2=/build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/slc7_amd64_gcc820/external/bz2lib/1.0.6-pafccj install --record /build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/pip-ucxcb1mi-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/pip-build-tyq1z3bp/numpy/
error: Bad exit status from /build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.U8gKBv (%build)


RPM build errors:
Bad exit status from /build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.U8gKBv (%build)

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-742fa0/2651/summary.html

cmsbuild · 2019-09-24T21:39:04Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmsdist#5241
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2656/console Started: 2019/09/24 23:40

cmsbuild · 2019-09-25T07:19:22Z

-1

Tested at: 92d000b

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-742fa0/2656/summary.html

I found follow errors while testing this PR

Failed tests: Build

Build:

I found compilation warning when building: See details on the summary page.

cmsbuild · 2019-09-25T07:41:01Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmsdist#5241
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2665/console Started: 2019/09/25 09:42

cmsbuild · 2019-09-25T17:37:39Z

-1

Tested at: 92d000b

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-742fa0/2665/summary.html

I found follow errors while testing this PR

Failed tests: UnitTests RelVals AddOn

Unit Tests:

I found errors in the following unit tests:

---> test testLLVMLite had ERRORS
---> test testPYDablooms had ERRORS
---> test testNumba had ERRORS
---> test testCxOracle had ERRORS
---> test testTheano had ERRORS
---> test runTheWhiteRabbit had ERRORS
---> test test_CreateFileLists had ERRORS
---> test test_PrepareInputDb had ERRORS
---> test testImports had ERRORS
---> test test_MpsWorkFlow had ERRORS
---> test condTestRegression had ERRORS
---> test testPyBind11ParameterSet had ERRORS
---> test TestFWCoreParameterSetDriver had ERRORS
---> test TestFWCoreConcurrencyDriver had ERRORS
---> test TestConfigDP had ERRORS
---> test testBottleneck had ERRORS
---> test EcalTPGFineGrainEBIdMap_O2O_test had ERRORS
---> test EcalLaser_O2O_test had ERRORS
---> test EcalTPGPhysicsConst_O2O_test had ERRORS
---> test EcalTPGWeightIdMap_O2O_test had ERRORS
---> test EcalDAQ_O2O_test had ERRORS
---> test EcalDCS_O2O_test had ERRORS
---> test EcalTPGFineGrainTowerEE_O2O_test had ERRORS
---> test EcalTPGFineGrainEBGroup_O2O_test had ERRORS
---> test EcalIntercalibConstants_O2O_test had ERRORS
---> test RunInfoStart_O2O_test had ERRORS
---> test SiStripDCS_O2O_test had ERRORS
---> test TestRunnerDataFormatsFWLite had ERRORS
---> test EcalTPGPedestals_O2O_test had ERRORS
---> test EcalTPGLutIdMap_O2O_test had ERRORS
---> test EcalTPGBadStripStatus_O2O_test had ERRORS
---> test EcalTPGFineGrainStripEE_O2O_test had ERRORS
---> test EcalTPGSpike_O2O_test had ERRORS
---> test EcalTPGCrystalStatus_O2O_test had ERRORS
---> test EcalTPGTowerStatus_O2O_test had ERRORS
---> test EcalTPGWeightGroup_O2O_test had ERRORS
---> test EcalTPGLinearizationConst_O2O_test had ERRORS
---> test EcalTPGLutGroup_O2O_test had ERRORS
---> test EcalTPGSlidingWindow_O2O_test had ERRORS

RelVals:

When I ran the RelVals I found an error in the following workflows:
25202.0 step2

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step2_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log

10224.0 step2

runTheMatrix-results/10224.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU+NanoFull_2017PU/step2_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU+NanoFull_2017PU.log

250202.181 step2

runTheMatrix-results/250202.181_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25/step2_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25.log

AddOn:

I found errors in the following addon tests:

Dr15Jones · 2019-09-25T18:15:54Z

One of the failures is from here:
https://github.com/cms-sw/cmssw/blob/master/Configuration/Applications/python/ConfigBuilder.py#L146

Matti says to fix the problem one needs to add universal_newlines=True to the Popen call.

cmsbuild · 2019-09-25T18:24:42Z

Pull request #28057 was updated. @smuzaffar, @Dr15Jones, @chayanit, @zhenhu, @kpedro88, @pgunnell, @franzoni, @cmsbuild, @fabiocos, @davidlange6 can you please check and sign again.

smuzaffar and others added 4 commits September 18, 2019 14:06

Python3: Changes needed to run runThePatrix with python2/3

79d3070

fix das-selected-lumis.py for python3

5b8a35b

use obj.lstrip() instead of string.lstrip(obj)

4a068da

cleanup: remove complex list(map()) use

92d000b

cmsbuild added comparison-pending core-pending operations-pending orp-pending pdmv-pending pending-signatures tests-pending upgrade-pending labels Sep 24, 2019

smuzaffar changed the title ~~From cmssw 11 0 x 2019 09 18 1100~~ [Python3][Devel] testing python3 in devel IBs Sep 24, 2019

smuzaffar mentioned this pull request Sep 24, 2019

[Python3][Devel] build root with python3 cms-sw/cmsdist#5241

Merged

cmsbuild added requires-external tests-started and removed tests-pending labels Sep 24, 2019

cmsbuild added tests-rejected and removed tests-started labels Sep 24, 2019

cmsbuild added tests-started and removed tests-rejected labels Sep 24, 2019

cmsbuild added tests-rejected and removed tests-started labels Sep 24, 2019

cmsbuild added tests-started and removed tests-rejected labels Sep 24, 2019

cmsbuild added tests-rejected and removed tests-started labels Sep 25, 2019

cmsbuild added tests-started and removed tests-rejected labels Sep 25, 2019

cmsbuild added tests-rejected and removed tests-started labels Sep 25, 2019

cmsbuild added tests-pending and removed requires-external tests-rejected labels Sep 25, 2019

cmsbuild added requires-external tests-rejected and removed tests-pending labels Sep 25, 2019

cmsbuild merged commit 92d000b into cms-sw:CMSSW_11_0_DEVEL_X Sep 26, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Python3][Devel] testing python3 in devel IBs #28057

[Python3][Devel] testing python3 in devel IBs #28057

smuzaffar commented Sep 24, 2019

cmsbuild commented Sep 24, 2019

cmsbuild commented Sep 24, 2019 •

edited

cmsbuild commented Sep 24, 2019

cmsbuild commented Sep 24, 2019

cmsbuild commented Sep 24, 2019 •

edited

cmsbuild commented Sep 24, 2019

cmsbuild commented Sep 24, 2019 •

edited

cmsbuild commented Sep 25, 2019

cmsbuild commented Sep 25, 2019 •

edited

cmsbuild commented Sep 25, 2019

Dr15Jones commented Sep 25, 2019

cmsbuild commented Sep 25, 2019

[Python3][Devel] testing python3 in devel IBs #28057

[Python3][Devel] testing python3 in devel IBs #28057

Conversation

smuzaffar commented Sep 24, 2019

cmsbuild commented Sep 24, 2019

cmsbuild commented Sep 24, 2019 • edited

cmsbuild commented Sep 24, 2019

cmsbuild commented Sep 24, 2019

cmsbuild commented Sep 24, 2019 • edited

cmsbuild commented Sep 24, 2019

cmsbuild commented Sep 24, 2019 • edited

cmsbuild commented Sep 25, 2019

cmsbuild commented Sep 25, 2019 • edited

cmsbuild commented Sep 25, 2019

Dr15Jones commented Sep 25, 2019

cmsbuild commented Sep 25, 2019

cmsbuild commented Sep 24, 2019 •

edited

cmsbuild commented Sep 24, 2019 •

edited

cmsbuild commented Sep 24, 2019 •

edited

cmsbuild commented Sep 25, 2019 •

edited