Pass thread count to gridpack run scripts. #17217
Conversation
Altered the ExternalLHEProducer to pass information from the cmsRun process to the script that launches the generators. This requires a small change to EDProducerBase so producers are informed of the number of threads in the current configuration. Generators that are ready to run multi-process / multi-threaded will use this extra argument; otherwise, this is ignored.
|
A new Pull Request was created by @bbockelm (Brian Bockelman) for CMSSW_9_0_X. It involves the following packages: FWCore/Framework @smuzaffar, @Dr15Jones, @perrozzi, @thuer, @cmsbuild, @govoni, @davidlange6 can you please review it and eventually sign? Thanks. cms-bot commands are listed here #13028 |
|
Thanks, this looks good to me (though I leave to others to comment whether this is the preferred way of passing the number of threads from the framework down to ExternalLHEProducer). Indeed only the two run_generic scripts would be expected to support multicore-capable gridpacks (and run_generic_tarball_cvmfs.sh is really the only one used in production these days.) |
|
Actually there's a second place where this should also be implemented. The "SUSY workflow" which can randomly select from multiple gridpacks bypasses the ExternalLHEProducer and calls the LHE generation scripts directly from within the generator interface. The function which calls the script is here And this is called from the actual EDFilter here |
|
For testing, the request which Dirk was testing earlier would be a good representative one where the gridpack should support multicore running to some level. https://cms-pdmv.cern.ch/mcm/requests?prepid=HIG-RunIISummer15wmLHEGS-00743&page=0&shown=131199 https://cms-pdmv.cern.ch/mcm/public/restapi/requests/get_test/HIG-RunIISummer15wmLHEGS-00743 |
|
+1 |
|
please test |
|
The tests are being triggered in jenkins. |
|
+1 |
cmsbuild
added
fully-signed
generators-approved
and removed
generators-pending
pending-signatures
labels
|
This pull request is fully signed and it will be integrated in one of the next CMSSW_9_0_X IBs after it passes the integration tests. This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @smuzaffar |
|
Note: given @bendavid's input above, there will be an additional commit on this PR to fixup |
|
-1 Tested at: c5c8ab4 You can see the results of the tests here: I found follow errors while testing this PR Failed tests: UnitTests
I found errors in the following unit tests: ---> test TestFWCoreIntegrationStandalone had ERRORS |
|
Comparison job queued. |
|
please test |
|
The tests are being triggered in jenkins. |
|
-1 Tested at: c5c8ab4 You can see the results of the tests here: I found follow errors while testing this PR Failed tests: UnitTests
I found errors in the following unit tests: ---> test testRecoMETMETProducers had ERRORS |
|
Comparison job queued. |
Altered the
ExternalLHEProducerto pass information from thecmsRunprocess to the script that launches the generators. This requires a small change toEDProducerBaseso producers are informed of the number of threads in the current configuration.Generators that are ready to run multi-process / multi-threaded will use this extra argument; otherwise, this is ignored.
@bendavid - does this reflect what we were discussing in email? Anything I missed? Do you have a suggested
cmsDrivercommand I could run to validate? I'm afraid I don't have deep knowledge on the generators...@sextonkennedy - this is a follow-up from our discussion in December with @Dr15Jones; Josh had suggested that the most expedient route would be for me to attempt the changes.