Skip to content

helper macros for install/removing data package#10430

Merged
smuzaffar merged 1 commit intoIB/CMSSW_16_1_X/masterfrom
share-data-pkg
Mar 18, 2026
Merged

helper macros for install/removing data package#10430
smuzaffar merged 1 commit intoIB/CMSSW_16_1_X/masterfrom
share-data-pkg

Conversation

@smuzaffar
Copy link
Copy Markdown
Contributor

This PR adds couple of helper macros which can use usd as

%post
%{install_shared_data_package}

%postun
%{uninstall_shared_data_package}

to make a copy of data package into share area and if all the references to shared data package are removed then also delete the shared package.

I have locally tested these macros using a testdata package (with some debug output ) and these work as expected

  • Building/Installing same realversion multiple time
  1. New relversion build
RPM installation stderr testdata:
Deploying: /build/muz/bld-time/w/share/external/testdata/v11/.links/arch-el8_amd64_gcc13-v11-62a15cf9e5e5e56c4aba77b9c299823c
mv bar /build/muz/bld-time/w/share/external/testdata/v11/bar
mv bla /build/muz/bld-time/w/share/external/testdata/v11/bla
mv foo /build/muz/bld-time/w/share/external/testdata/v11/foo
  1. Rebuilding again i.e. realversion remain the same
RPM installation stderr testdata:
Deploying: /build/muz/bld-time/w/share/external/testdata/v11/.links/arch-el8_amd64_gcc13-v11-032866d66313672c6d52d27be3983aab
Already exists /build/muz/bld-time/w/share/external/testdata/v11/bar
Already exists /build/muz/bld-time/w/share/external/testdata/v11/bla
Already exists /build/muz/bld-time/w/share/external/testdata/v11/foo
  1. Rebuilding again i.e. realversion remain the same
RPM installation stderr testdata:
Deploying: /build/muz/bld-time/w/share/external/testdata/v11/.links/arch-el8_amd64_gcc13-v11-1075c3e4c4537645d856fc1d8351c0f9
Already exists /build/muz/bld-time/w/share/external/testdata/v11/bar
Already exists /build/muz/bld-time/w/share/external/testdata/v11/bla
Already exists /build/muz/bld-time/w/share/external/testdata/v11/foo

For three different builds of same data package version, we end up with a single copy of it in share path.

  • Removing installed shared data package
> cmspkg -a el8_amd64_gcc13 remove -y --delete-dir external+testdata+v11-1075c3e4c4537645d856fc1d8351c0f9
Removing package external+testdata+v11-1075c3e4c4537645d856fc1d8351c0f9
Uninstall /build/muz/bld-time/w/share/external/testdata/v11/.links/-arch-el8_amd64_gcc13-v11-1075c3e4c4537645d856fc1d8351c0f9
Not cleaning up shared installation
Removed external+testdata+v11-1075c3e4c4537645d856fc1d8351c0f9
> cmspkg -a el8_amd64_gcc13 remove -y --delete-dir external+testdata+v11-032866d66313672c6d52d27be3983aab
Removing package external+testdata+v11-032866d66313672c6d52d27be3983aab
Uninstall /build/muz/bld-time/w/share/external/testdata/v11/.links/-arch-el8_amd64_gcc13-v11-032866d66313672c6d52d27be3983aab
Not cleaning up shared installation
Removed external+testdata+v11-032866d66313672c6d52d27be3983aab
> cmspkg -a el8_amd64_gcc13 remove -y --delete-dir external+testdata+v11-62a15cf9e5e5e56c4aba77b9c299823c
Removing package external+testdata+v11-62a15cf9e5e5e56c4aba77b9c299823c
Uninstall /build/muz/bld-time/w/share/external/testdata/v11/.links/-arch-el8_amd64_gcc13-v11-62a15cf9e5e5e56c4aba77b9c299823c
Uninstall shared: /build/muz/bld-time/w/share/external/testdata/v11
Removed external+testdata+v11-62a15cf9e5e5e56c4aba77b9c299823c

@cmsbuild
Copy link
Copy Markdown
Contributor

A new Pull Request was created by @smuzaffar for branch IB/CMSSW_16_1_X/master.

@akritkbehera, @cmsbuild, @iarspider, @raoatifshad, @smuzaffar can you please review it and eventually sign? Thanks.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.
cms-bot commands are listed here

@cmsbuild
Copy link
Copy Markdown
Contributor

cmsbuild commented Mar 18, 2026

cms-bot internal usage

@smuzaffar
Copy link
Copy Markdown
Contributor Author

please test

though as no package use these macros to nothing should build/rebuild

@smuzaffar
Copy link
Copy Markdown
Contributor Author

+externals

as expected, nothing was re-build due to this change. These are just helper macros which data packages can use in future

@smuzaffar smuzaffar merged commit 4ba47e5 into IB/CMSSW_16_1_X/master Mar 18, 2026
7 of 8 checks passed
@cmsbuild
Copy link
Copy Markdown
Contributor

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_16_1_X/master IBs after it passes the integration tests. This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @ftenchini, @mandrenguyen (and backports should be raised in the release meeting by the corresponding L2)

@cmsbuild
Copy link
Copy Markdown
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5224a7/52075/summary.html
COMMIT: 92efdf7
CMSSW: CMSSW_16_1_X_2026-03-17-2300/el8_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/10430/52075/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5224a7/52075/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5224a7/52075/git-merge-result

Comparison Summary

Summary:

  • You potentially added 24 lines to the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 1 differences found in the comparisons
  • DQMHistoTests: Total files compared: 53
  • DQMHistoTests: Total histograms compared: 4185208
  • DQMHistoTests: Total failures: 3
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4185185
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 52 files compared)
  • Checked 227 log files, 198 edm output root files, 53 DQM output files
  • TriggerResults: no differences found

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants