scripts for running MPI jobs in the CMSSW environment [12.1.x]

[fwyzard]

PR description:

Add two scripts for running MPI jobs in the CMSSW environment:

• cmsenv_mpirun is a wrapper around mpirun; it uses cmsenv_orted as the remote "agent" and sets its verbose and release flags according to the other MPI flags;
• cmsenv_orted is a wrapper around orted; it loads the CMSSW environment and passes it to the requested command(s).

PR validation:

Tested with testMPI:

$ cmsenv_mpirun --verbose -H localhost -np 1 $CMSSW_RELEASE_BASE/test/$SCRAM_ARCH/testMPI : -H fu-c2a02-37-02.cms -np 3 -oversubscribe $CMSSW_RELEASE_BASE/test/$SCRAM_ARCH/testMPI

        Number of Processes 4
        Number of workSplit First 7
        Number of workSplit Second 0

                (1) Non-Blocking Scatter 


        =============================================================
        || Func ||  Scatter/Send ||   Gather/Receive  || Number Run||
        =============================================================
        ||  1   ||     2.0788    ||        0.1752     ||      5    ||
        =============================================================

if this PR is a backport please specify the original PR and why you need to backport that PR:

Backport of #36109

[fwyzard]

backport #36109

[fwyzard]

please test

[cmsbuild]

A new Pull Request was created by @fwyzard (Andrea Bocci) for CMSSW_12_1_X.

It involves the following packages:

• HeterogeneousCore/MPICore (heterogeneous)

@makortel, @fwyzard can you please review it and eventually sign? Thanks.
@makortel, @rovere this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

• Backported from scripts for running MPI jobs in the CMSSW environment #36109

[fwyzard]

+heterogeneous

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_1_X IBs after it passes the integration tests and once validation in the development release cycle CMSSW_12_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18f225/20510/summary.html
COMMIT: 9a3c39a
CMSSW: CMSSW_12_1_X_2021-11-12-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/36110/20510/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 18 differences found in the comparisons
• DQMHistoTests: Total files compared: 42
• DQMHistoTests: Total histograms compared: 2901440
• DQMHistoTests: Total failures: 0
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2901418
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 41 files compared)
• Checked 177 log files, 37 edm output root files, 42 DQM output files
• TriggerResults: no differences found

[qliphy]

+1