Update various low-level packages e.g. pip, setuptools etc. #7696

iarspider · 2022-03-16T11:05:54Z

…; update curl to 7.79.0; update pip to 22.0.4

iarspider · 2022-03-16T11:06:06Z

pleas test

cmsbuild · 2022-03-16T11:06:11Z

A new Pull Request was created by @iarspider for branch IB/CMSSW_12_4_X/master.

@cmsbuild, @smuzaffar, @iarspider can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @qliphy you are the release manager for this.
cms-bot commands are listed here

…check commands

smuzaffar · 2022-03-16T11:48:50Z

please test

cmsbuild · 2022-03-16T11:49:04Z

Pull request #7696 was updated.

cmsbuild · 2022-03-16T13:50:41Z

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-45ef5a/23158/summary.html
COMMIT: f1037a7
CMSSW: CMSSW_12_4_X_2022-03-15-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/7696/23158/install.sh to create a dev area with all the needed externals and cmssw changes.

External Build

I found compilation error when building:

---------- -------------------
pip        22.0.4
setuptools 60.9.3.post20220316
+ '%{PipInstallCommand}'
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.iHlzdY: line 65: fg: no job control
error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.iHlzdY (%build)


RPM build errors:
Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.iHlzdY (%build)

smuzaffar · 2022-03-16T14:10:24Z

please test

cmsbuild · 2022-03-16T14:15:11Z

Pull request #7696 was updated.

smuzaffar · 2022-03-16T22:05:43Z

please test for slc7_aarch64_gcc11

smuzaffar · 2022-03-16T22:06:37Z

please test for slc7_ppc64le_gcc11

cmsbuild · 2022-03-17T03:18:11Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-45ef5a/23163/summary.html
COMMIT: 00a2eba
CMSSW: CMSSW_12_4_X_2022-03-15-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/7696/23163/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@cms-sw Update aiohttp to 3.8.1 #7690

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-45ef5a/23163/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-45ef5a/23163/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test import-numexpr had ERRORS
---> test import-tables had ERRORS
---> test testNumExpr had ERRORS
---> test testTables had ERRORS
and more ...

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3695377
DQMHistoTests: Total failures: 11
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3695344
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 204 log files, 45 edm output root files, 49 DQM output files
TriggerResults: found differences in 1 / 48 workflows

cmsbuild · 2022-03-17T07:32:44Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-45ef5a/23170/summary.html
COMMIT: 00a2eba
CMSSW: CMSSW_12_4_X_2022-03-15-2300/slc7_ppc64le_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/7696/23170/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-45ef5a/23170/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-45ef5a/23170/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test testFWCoreUtilities had ERRORS
---> test import-numexpr had ERRORS
---> test import-tables had ERRORS
---> test testNumExpr had ERRORS
and more ...

cmsbuild · 2022-03-17T07:44:13Z

-1

Failed Tests: UnitTests RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-45ef5a/23169/summary.html
COMMIT: 00a2eba
CMSSW: CMSSW_12_4_X_2022-03-15-2300/slc7_aarch64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/7696/23169/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-45ef5a/23169/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-45ef5a/23169/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test TestFWCoreServicesDriver had ERRORS
---> test testFWCoreUtilities had ERRORS
---> test import-numexpr had ERRORS
---> test import-tables had ERRORS
and more ...

RelVals

4.53

----- Begin Fatal Exception 17-Mar-2022 07:51:25 CET-----------------------
An exception of category 'Vertex' occurred while
   [0] Processing  Event run: 194533 lumi: 329 event: 462355458 stream: 0
   [1] Running path 'dqmofflineOnPAT_1_step'
   [2] Prefetching for module SingleTopTChannelLeptonDQM_miniAOD/'singleTopElectronMediumDQM_miniAOD'
   [3] Prefetching for module PATMuonSlimmer/'slimmedMuons'
   [4] Prefetching for module PATMuonSelector/'selectedPatMuons'
   [5] Prefetching for module PATMuonProducer/'patMuons'
   [6] Prefetching for module MuonProducer/'muons'
   [7] Prefetching for module PFProducer/'particleFlowTmp'
   [8] Prefetching for module PFBlockProducer/'particleFlowBlock'
   [9] Prefetching for module PFElecTkProducer/'pfTrackElec'
   [10] Prefetching for module PFConversionProducer/'pfConversions'
   [11] Calling method for module ConversionProducer/'allConversions'
Exception Message:
Refitted track not found in list
----- End Fatal Exception -------------------------------------------------

140.56

----- Begin Fatal Exception 17-Mar-2022 08:00:46 CET-----------------------
An exception of category 'Vertex' occurred while
   [0] Processing  Event run: 326479 lumi: 7 event: 1579493 stream: 0
   [1] Running path 'dqmoffline_8_step'
   [2] Prefetching for module SMPDQM/'SMPDQM'
   [3] Prefetching for module MuonProducer/'muons'
   [4] Prefetching for module PFProducer/'particleFlowTmp'
   [5] Prefetching for module PFBlockProducer/'particleFlowBlock'
   [6] Prefetching for module PFElecTkProducer/'pfTrackElec'
   [7] Prefetching for module PFConversionProducer/'pfConversions'
   [8] Calling method for module ConversionProducer/'allConversions'
Exception Message:
Refitted track not found in list
----- End Fatal Exception -------------------------------------------------

136.874

----- Begin Fatal Exception 17-Mar-2022 08:07:23 CET-----------------------
An exception of category 'Vertex' occurred while
   [0] Processing  Event run: 319450 lumi: 76 event: 106007323 stream: 0
   [1] Running path 'dqmoffline_10_step'
   [2] Prefetching for module SMPDQM/'SMPDQM'
   [3] Prefetching for module MuonProducer/'muons'
   [4] Prefetching for module PFProducer/'particleFlowTmp'
   [5] Prefetching for module PFBlockProducer/'particleFlowBlock'
   [6] Prefetching for module PFElecTkProducer/'pfTrackElec'
   [7] Prefetching for module PFConversionProducer/'pfConversions'
   [8] Calling method for module ConversionProducer/'allConversions'
Exception Message:
Refitted track not found in list
----- End Fatal Exception -------------------------------------------------

smuzaffar · 2022-03-17T07:57:17Z

please test

cmsbuild · 2022-03-17T07:57:23Z

Pull request #7696 was updated.

cmsbuild · 2022-03-17T16:18:46Z

Pull request #7696 was updated.

iarspider · 2022-03-17T16:32:49Z

please test

cmsbuild · 2022-03-18T01:58:06Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-45ef5a/23185/summary.html
COMMIT: 9387eb6
CMSSW: CMSSW_12_4_X_2022-03-17-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/7696/23185/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test test_edmPickEvents had ERRORS

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3695377
DQMHistoTests: Total failures: 17
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3695338
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 204 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

smuzaffar · 2022-03-18T05:55:19Z

+externals
looks good

cmsbuild · 2022-03-18T05:55:40Z

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_12_4_X/master IBs (but tests are reportedly failing). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

iarspider and others added 2 commits March 16, 2022 08:44

Update setuptools to 60.9.3; drop py3-nose, py3-hyperas, py3-hyperopt…

5c93944

…; update curl to 7.79.0; update pip to 22.0.4

Remove bootstrap step from setuptools spec

4264be9

cmsbuild added externals-pending orp-pending pending-signatures tests-pending labels Mar 16, 2022

updated build-with-pip.file to allow overriding pip-install, pip-dep-…

f1037a7

…check commands

cmsbuild added tests-started and removed tests-pending labels Mar 16, 2022

cmsbuild added tests-rejected and removed tests-started labels Mar 16, 2022

typo: added missing PipInstallCommand macro

00a2eba

cmsbuild added tests-started and removed tests-rejected labels Mar 16, 2022

cmsbuild added tests-rejected and removed tests-started labels Mar 17, 2022

smuzaffar added 2 commits March 17, 2022 08:51

updated numexpr and fixed the build-with-pip

47e57b5

added PipDepsMatch and PipInstallCommand macros

079d12d

cmsbuild added tests-started and removed tests-rejected labels Mar 17, 2022

iarspider changed the title ~~Update setuptools, pip, curl~~ Update some low-level packages Mar 17, 2022

Update pcre2 to 10.36

9387eb6

cmsbuild added tests-pending and removed tests-started labels Mar 17, 2022

cmsbuild added tests-started and removed tests-pending labels Mar 17, 2022

smuzaffar changed the title ~~Update some low-level packages~~ Update various low-level packages e.g. pip, setuptools etc. Mar 17, 2022

cmsbuild added tests-rejected and removed tests-started labels Mar 18, 2022

smuzaffar merged commit 539523f into IB/CMSSW_12_4_X/master Mar 18, 2022

cmsbuild added externals-approved fully-signed and removed externals-pending pending-signatures labels Mar 18, 2022

This was referenced Mar 18, 2022

Update curl to 7.79.0 #7689

Closed

Update pip to 21.3.1 #7662

Closed

Update setuptools to 58.2.0; drop nose, hyperas, hyperopt #7400

Closed

Use case-insensitive comparision for checking dependencies of python packages #7534

Closed

This was referenced Mar 19, 2022

Update swig to 4.0.2 #7699

Merged

Dynamic Reduction Network for photon energy regression in miniAOD using the SonicTriton service cms-sw/cmssw#37134

Merged

iarspider deleted the update_setuptools_pip_curl branch March 21, 2022 07:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update various low-level packages e.g. pip, setuptools etc. #7696

Update various low-level packages e.g. pip, setuptools etc. #7696

iarspider commented Mar 16, 2022

iarspider commented Mar 16, 2022

cmsbuild commented Mar 16, 2022

smuzaffar commented Mar 16, 2022

cmsbuild commented Mar 16, 2022

cmsbuild commented Mar 16, 2022

smuzaffar commented Mar 16, 2022

cmsbuild commented Mar 16, 2022

smuzaffar commented Mar 16, 2022

smuzaffar commented Mar 16, 2022

cmsbuild commented Mar 17, 2022

cmsbuild commented Mar 17, 2022

cmsbuild commented Mar 17, 2022

smuzaffar commented Mar 17, 2022

cmsbuild commented Mar 17, 2022

cmsbuild commented Mar 17, 2022

iarspider commented Mar 17, 2022

cmsbuild commented Mar 18, 2022

smuzaffar commented Mar 18, 2022

cmsbuild commented Mar 18, 2022

Update various low-level packages e.g. pip, setuptools etc. #7696

Update various low-level packages e.g. pip, setuptools etc. #7696

Conversation

iarspider commented Mar 16, 2022

iarspider commented Mar 16, 2022

cmsbuild commented Mar 16, 2022

smuzaffar commented Mar 16, 2022

cmsbuild commented Mar 16, 2022

cmsbuild commented Mar 16, 2022

External Build

smuzaffar commented Mar 16, 2022

cmsbuild commented Mar 16, 2022

smuzaffar commented Mar 16, 2022

smuzaffar commented Mar 16, 2022

cmsbuild commented Mar 17, 2022

Unit Tests

Comparison Summary

cmsbuild commented Mar 17, 2022

Unit Tests

cmsbuild commented Mar 17, 2022

Unit Tests

RelVals

smuzaffar commented Mar 17, 2022

cmsbuild commented Mar 17, 2022

cmsbuild commented Mar 17, 2022

iarspider commented Mar 17, 2022

cmsbuild commented Mar 18, 2022

Unit Tests

Comparison Summary

smuzaffar commented Mar 18, 2022

cmsbuild commented Mar 18, 2022