Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify caching of CLHEP::HepRandomEngine in digitizers #22874

Merged
merged 2 commits into from Apr 13, 2018

Conversation

makortel
Copy link
Contributor

@makortel makortel commented Apr 6, 2018

The explicit stream-based caching was introduced in #2392 at the time the the MixingModule was still a "one" module. Now that the MixingModule is a stream producer, there is no need to do the caching explicitly (=vectors indexed by the stream ID). A simple member variable is enough, and it is safe because framework runs always the same instance of a stream module in a given stream.

The caching is further simplified to always set the cache variable in initializeEvent() and set it back to nullptr in finalizeEvent() (to prevent the use of the engine outside event, e.g. begin/end lumi/run).

Resolves #22712.

Tested in CMSSW_10_1_0 (rebased on top of CMSSW_10_2_X_2018-04-10-2300), no changes expected.

@wddgit @Dr15Jones

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 6, 2018

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 6, 2018

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 6, 2018

A new Pull Request was created by @makortel (Matti Kortelainen) for master.

It involves the following packages:

SimCalorimetry/CastorSim
SimCalorimetry/EcalSimProducers
SimCalorimetry/HGCalSimProducers
SimCalorimetry/HcalSimProducers
SimCalorimetry/HcalTestBeam
SimFastTiming/FastTimingCommon
SimTracker/SiPixelDigitizer
SimTracker/SiStripDigitizer

@cmsbuild, @civanch, @kpedro88, @mdhildreth can you please review it and eventually sign? Thanks.
@echabert, @vandreev11, @sethzenz, @mariadalfonso, @GiacomoSguazzoni, @kpedro88, @lgray, @cseez, @prolay, @dkotlins, @pfs, @threus, @VinInn, @ebrondol, @argiro, @rovere, @dgulhan, @mmusich this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

@makortel
Copy link
Contributor Author

makortel commented Apr 6, 2018

@cmsbuild, please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 6, 2018

The tests are being triggered in jenkins.

@civanch
Copy link
Contributor

civanch commented Apr 6, 2018

@makortel , @Dr15Jones ,as I understand the logic of random numbers, in SIM step both for FullSim and FastSim the RandomNumberService is caled once per event to define random number generator for given event (Inside current thread). In FastSim this generator is distributed a a parameter of all methods to all called sub-classes. In the case of FullSim/Geant4 the random engine is set per event to as a CLHEP engine in the given thread. These both variants are transparent and seems to be correct from the design point of view : method of definition of the generator is fully disconnected with FullSim and FastSim. The ovehead of access to the service per event is minimal.

At the DIGI/Mixing steps there is a sequence of producers, in each we need to access the service. There may be more overhead, however, to me having engine definded per event is indeed more safe than keeping one engine stored after the first call. I afread that the method proposed by Matti may provide completely non-reproducible code, because event sequence in a thread is not reproducible.

@makortel
Copy link
Contributor Author

makortel commented Apr 6, 2018

@civanch Technically the current "StreamID-indexed vector" caching, and the caching done in this PR, are safe, and produce exactly the same results. (motivation for this PR being clearer code so that one does not have to wonder why there is a vector)

Would your preference be then to set the cache in each call to initializeEvent() (and reset to nullptr in finalizeEvent()? I agree that would be clearer to a reader who would then not have to think/know about the exact details of random number service.

@civanch
Copy link
Contributor

civanch commented Apr 6, 2018

@makortel , I would think a good idea about initializeEvent() and I would do nothing in finelizeEvent().

@kpedro88
Copy link
Contributor

please test
(seems stuck)

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 10, 2018

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/27403/console Started: 2018/04/10 09:11

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Pull request #22874 was updated. @cmsbuild, @civanch, @kpedro88, @mdhildreth can you please check and sign again.

@makortel
Copy link
Contributor Author

Rebased on top of CMSSW_10_2_X_2018-04-10-2300 and simplified the caching further along #22874 (comment).

@civanch I dediced in the end to set the cache variable back to nullptr in finalizeEvent() to prevent its use outside of Event (i.e. in begin/end lumi/run).

@makortel
Copy link
Contributor Author

@cmsbuild, please test with workflow 250202.0,250202.17,250202.18,250402.0

Let's include premixing workflows as well to demonstrate that they continue to work (as some parts of digitizer code is called from premixing)

@makortel makortel changed the title Replace explicit stream-based caching of CLHEP::HepRandomEngine with implicit one in digitizers Simplify caching of CLHEP::HepRandomEngine in digitizers Apr 11, 2018
@makortel
Copy link
Contributor Author

@cmsbuild, please test workflow 250202.0,250202.17,250202.18,250402.0

Maybe some day I'll remember the syntax on the top of my head...

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 11, 2018

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/27433/console Started: 2018/04/11 14:19

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22874/27433/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

  • /build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22874/250202.18_TTbar_13UP18+TTbar_13UP18+DIGIPRMXUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25
  • /build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22874/250402.0_TTbar_13+FS_TTbar_13_PRMXUP15_PU25+HARVESTUP15FS+MINIAODMCUP15FS

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 29
  • DQMHistoTests: Total histograms compared: 2505375
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2505198
  • DQMHistoTests: Total skipped: 176
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.969999999943 KiB( 23 files compared)
  • Checked 119 log files, 9 edm output root files, 29 DQM output files

@civanch
Copy link
Contributor

civanch commented Apr 11, 2018

+1

@kpedro88
Copy link
Contributor

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

@fabiocos
Copy link
Contributor

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants