Add test for missing Provides #1936

iarspider · 2023-02-09T15:27:44Z

No description provided.

cmsbuild · 2023-02-09T15:28:07Z

A new Pull Request was created by @iarspider for branch master.

@cmsbuild, @smuzaffar, @aandvalenzuela, @iarspider can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.
cms-bot commands are listed here

cmsbuild · 2023-02-09T16:56:24Z

Pull request #1936 was updated.

smuzaffar · 2023-02-09T17:30:02Z

thanks @iarspider . Can you please

I do not understand the purpose of https://github.com/cms-sw/cms-bot/pull/1936/files#diff-51dd032b1bff11f296c5943e95c62abe50c617f7a5e23511fd0d94c096b4feb9R33-R37 , we should be copying the lib/bin from the CMSSW area built by the PR . I think it should be under $WORKSPACE/$CMSSW_RELEASE (but double check please)
Only git clone cmsdist/pkgtools if not already available in $WORKSPACE. And if you have to clone them then use --depth 1 option too :-)
replace the hard coded x86_64 and el8_amd64_gcc11 with uname and $SCRAM_ARCH
https://github.com/cms-sw/cms-bot/pull/1936/files#diff-51dd032b1bff11f296c5943e95c62abe50c617f7a5e23511fd0d94c096b4feb9R23 will not work, you need to copy the cmssw-pr-package.spec file from cms-bot/pr_testing directory
No need of https://github.com/cms-sw/cms-bot/pull/1936/files#diff-51dd032b1bff11f296c5943e95c62abe50c617f7a5e23511fd0d94c096b4feb9R7 instead move the execution of test to https://github.com/cms-sw/cms-bot/pull/1936/files#diff-5e02a17340b02f75de80cf0d3a45e66970f1884b0a5cdf726ef1223846c95af3R1035 (i.e we run it there are not build errors) and run something like the following

if ${CMS_BOT_DIR}/pr_testing/test-cmssw-provides.sh ${SCRAM_ARCH} ${PKGTOOLS_BRANCH} ${CMSDIST_TAG} ${BUILD_DIR} ${WEEK_NUM} ${CMSSW_RELEASE} >$WORKSPACE/rpm-deps-checks.log 2>&1 ; then
  echo "SCRAM_RPM_DEPS;OK,Package dependency,See Log,rpm-deps-checks.log" >> ${RESULTS_DIR}/build.txt
else
  echo "SCRAM_RPM_DEPS;ERROR,Package dependency,See Log,rpm-deps-checks.log" >> ${RESULTS_DIR}/build.txt
fi

Have you tested this script locally? path in https://github.com/cms-sw/cms-bot/pull/1936/files#diff-51dd032b1bff11f296c5943e95c62abe50c617f7a5e23511fd0d94c096b4feb9R39 does not look correct, you are using PKGTOOLS and CMSDIST while the git clone creates pkgtools and cmsdist. Also you need to run cmsBuild once to create the bootstrap area and then 2nd time to build cmssw-pr-package. Can you please test it locally first , the var/lib/rpm should to copied to BUILD/$SCRAM_ARCH/var/lib/rpm

cmsbuild · 2023-02-10T10:40:20Z

Pull request #1936 was updated.

cmsbuild · 2023-02-10T10:45:44Z

Pull request #1936 was updated.

pr_testing/test-cmssw-provides.sh

smuzaffar · 2023-02-11T10:25:51Z

you need to run cmsBuild once to create the bootstrap area

Looks like this is not done . You need to run cmsBuild once which should bootstrap the area and after then you should copy the var/lib/rpm contents. It is enough to run the first cmsBuild command to build/install cms-common

pr_testing/test-cmssw-provides.sh

iarspider · 2023-02-11T10:29:45Z

you need to run cmsBuild once to create the bootstrap area

Looks like this is not done . You need to run cmsBuild once which should bootstrap the area and after then you should copy the var/lib/rpm contents. It is enough to run the first cmsBuild command to build/install cms-common

Still working on this, didn't yet implement all the comments.

smuzaffar · 2023-02-11T10:36:32Z

Still working on this, didn't yet implement all the comments.

ah ok

cmsbuild · 2023-02-13T12:54:59Z

Pull request #1936 was updated.

iarspider · 2023-02-13T14:12:41Z

@smuzaffar this PR is ready for review.

smuzaffar · 2023-02-14T23:29:52Z

test parameters:

addpkg = FWCore

smuzaffar · 2023-02-14T23:30:20Z

please test

looks good, lets test it with out cmsdist PR first

cmsbuild · 2023-03-03T16:58:48Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e3bdf5/31040/summary.html
COMMIT: 6e936ee
CMSSW: CMSSW_13_1_X_2023-03-02-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cms-bot/1936/31040/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@fwyzard [BuildRules] Enable alpaka ROCm backend cmsdist#8346

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e3bdf5/31040/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e3bdf5/31040/git-merge-result

Comparison Summary

Summary:

You potentially added 6 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3529699
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 28
DQMHistoTests: Total successes: 3529649
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 6.727 KiB( 48 files compared)
DQMHistoSizes: changed ( 20834.0,... ): 0.961 KiB MTD/ETL
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: no differences found

smuzaffar · 2023-03-04T10:55:34Z

please test

test without cmsdist PR

cmsbuild · 2023-03-04T13:29:30Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e3bdf5/31068/summary.html
COMMIT: 6e936ee
CMSSW: CMSSW_13_1_X_2023-03-03-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cms-bot/1936/31068/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 3 lines from the logs
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3529727
DQMHistoTests: Total failures: 7
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3529698
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: no differences found

smuzaffar · 2023-03-06T09:38:18Z

@iarspider , dependency check failed again https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e3bdf5/31068/rpm-deps-checks.log

cmsbuild · 2023-03-06T09:40:52Z

Pull request #1936 was updated.

iarspider · 2023-03-06T15:04:21Z

please test

smuzaffar · 2023-03-06T16:08:10Z

+externals
looks good now, there is some cleanup we can do which I will do later today

cmsbuild · 2023-03-06T16:08:37Z

This pull request is fully signed and it will be integrated in one of the next master IBs after it passes the integration tests. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild · 2023-03-06T17:36:05Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e3bdf5/31090/summary.html
COMMIT: 97d11e1
CMSSW: CMSSW_13_1_X_2023-03-06-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cms-bot/1936/31090/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 5 lines to the logs
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3529727
DQMHistoTests: Total failures: 7
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3529698
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: no differences found

Add test for missing Provides

35964d2

iarspider requested a review from smuzaffar February 9, 2023 15:27

cmsbuild added orp-pending pending-signatures externals-pending tests-pending labels Feb 9, 2023

Move spec file here

73a69ed

Changes from review (1/?)

e543a37

Change permission

1547214

smuzaffar reviewed Feb 11, 2023

View reviewed changes

pr_testing/test-cmssw-provides.sh Outdated Show resolved Hide resolved

smuzaffar reviewed Feb 11, 2023

View reviewed changes

pr_testing/test-cmssw-provides.sh Outdated Show resolved Hide resolved

smuzaffar reviewed Feb 11, 2023

View reviewed changes

pr_testing/test-cmssw-provides.sh Outdated Show resolved Hide resolved

smuzaffar reviewed Feb 11, 2023

View reviewed changes

pr_testing/test-cmssw-provides.sh Outdated Show resolved Hide resolved

iarspider marked this pull request as draft February 11, 2023 21:38

Changes from review (2/?)

aad590f

iarspider marked this pull request as ready for review February 13, 2023 14:09

cmsbuild added tests-started and removed tests-pending labels Feb 14, 2023

cmsbuild added tests-approved and removed tests-started labels Mar 3, 2023

cmsbuild added tests-started and removed requires-external tests-approved labels Mar 4, 2023

cmsbuild added tests-approved and removed tests-started labels Mar 4, 2023

smuzaffar mentioned this pull request Mar 4, 2023

[Do not merge] Testing bot cms-sw/cmsdist#8330

Closed

Update test-cmssw-provides.sh

97d11e1

cmsbuild added tests-pending and removed tests-approved labels Mar 6, 2023

cmsbuild added tests-started and removed tests-pending labels Mar 6, 2023

smuzaffar merged commit 57b5577 into master Mar 6, 2023

cmsbuild added fully-signed externals-approved and removed pending-signatures externals-pending labels Mar 6, 2023

cmsbuild added tests-approved and removed tests-started labels Mar 6, 2023

smuzaffar deleted the cmssw-pr-package branch March 6, 2023 20:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add test for missing Provides #1936

Add test for missing Provides #1936

iarspider commented Feb 9, 2023

cmsbuild commented Feb 9, 2023

cmsbuild commented Feb 9, 2023

smuzaffar commented Feb 9, 2023 •

edited

cmsbuild commented Feb 10, 2023

cmsbuild commented Feb 10, 2023

smuzaffar commented Feb 11, 2023

iarspider commented Feb 11, 2023

smuzaffar commented Feb 11, 2023

cmsbuild commented Feb 13, 2023

iarspider commented Feb 13, 2023

smuzaffar commented Feb 14, 2023 •

edited

smuzaffar commented Feb 14, 2023

cmsbuild commented Mar 3, 2023

smuzaffar commented Mar 4, 2023

cmsbuild commented Mar 4, 2023

smuzaffar commented Mar 6, 2023

cmsbuild commented Mar 6, 2023

iarspider commented Mar 6, 2023

smuzaffar commented Mar 6, 2023

cmsbuild commented Mar 6, 2023

cmsbuild commented Mar 6, 2023

Add test for missing Provides #1936

Add test for missing Provides #1936

Conversation

iarspider commented Feb 9, 2023

cmsbuild commented Feb 9, 2023

cmsbuild commented Feb 9, 2023

smuzaffar commented Feb 9, 2023 • edited

cmsbuild commented Feb 10, 2023

cmsbuild commented Feb 10, 2023

smuzaffar commented Feb 11, 2023

iarspider commented Feb 11, 2023

smuzaffar commented Feb 11, 2023

cmsbuild commented Feb 13, 2023

iarspider commented Feb 13, 2023

smuzaffar commented Feb 14, 2023 • edited

smuzaffar commented Feb 14, 2023

cmsbuild commented Mar 3, 2023

Comparison Summary

smuzaffar commented Mar 4, 2023

cmsbuild commented Mar 4, 2023

Comparison Summary

smuzaffar commented Mar 6, 2023

cmsbuild commented Mar 6, 2023

iarspider commented Mar 6, 2023

smuzaffar commented Mar 6, 2023

cmsbuild commented Mar 6, 2023

cmsbuild commented Mar 6, 2023

Comparison Summary

smuzaffar commented Feb 9, 2023 •

edited

smuzaffar commented Feb 14, 2023 •

edited