Robust random seed plan for digitizers

[rlcee]

Currently only geant (Mu2eG4) module runs multi-threaded. In order to keep random numbers reproducible, each event has its random number reseeded by the event number (plus salt?). So whichever thread receives an event, the simulation will still reproduce. If the events are then written out, and the next stage reads the events back in, art causes the events to enter the new path sorted by event number, and therefore if random numbers are used by single-threaded modules in this second stage, the result is still reproducible. I think this is what is happening in our current procedures.

In the case that there are modules which use random numbers and are in the same job as a multi-threaded geant and downstream of geant in the path, then those modules will get events in a semi-random order, their random chains will be unpredictable, and the result of these modules will be not be reproducible. A notable case is if the digitizers, which use random numbers, would be active in the same job as a multi-threaded Mu2eG4. Further note that ceSimReco does run digitizers in the same job as geant, but it uses single-threaded geant, so will not have this problem.

A potential solution is to run the digitizers with random numbers seeded by the event number, like the geant module. This might be able to be implemented as a feature of the random number service.

[pavel1murat]

an issue with the RN stream initialization exists even in a single-threaded environment. Modules using RN's and getting their initialized RN streams from the SeedService "interfere" with each other, so the RN's used by a module, in an obscure way, depend on the presence of other users of the SeedService

In a single-threaded mode, this issue could be addressed by making the starting RN stream for a given module a function of the module label, instead of the order in which art initializes modules

[rlcee]

Initalizing CLHEP HepJamesRandom (setSeed method) 1M times implies one intialization takes about 26e-6s. It involves several times 10k arithmetic operations. A ceSimReco event takes 0.2s, so intializing 20 random engines every event would add 0.25% to the CPU time.

[brownd1978]

ceSimReco isn't a production job, more relevant is pure signal digitization, which is ~0.1 second/event. Mixing takes a few seconds/event. So again the same conclusion. As long as we can avoid unnecessarily creating seeds for modules which don't need them this doesn't seem a problem.

…

On Tue, Oct 18, 2022 at 12:42 PM Ray Culbertson ***@***.***> wrote: Initalizing CLHEP HepJamesRandom (setSeed method) 1M times implies one intialization takes about 26e-6s. It involves several times 10k arithmetic operations. A ceSimReco event takes 0.2s, so intializing 20 random engines every event would add 0.25% to the CPU time. — Reply to this email directly, view it on GitHub <#849 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABAH572V5MNQHWLLFZSPVJTWD34SPANCNFSM6AAAAAAQK7TFBU> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- David Brown ***@***.*** Office Phone (510) 486-7261 Lawrence Berkeley National Lab M/S 50R5008 (50-6026C) Berkeley, CA 94720

[kutschke]

Hi Ray, Did you do this with a single 32 bit seed? When you get a chance can you remeasure using seeds of length 2, 3, 4 longs? Rob From: Ray Culbertson ***@***.***> Reply-To: Mu2e/Offline ***@***.***> Date: Tuesday, October 18, 2022 at 2:42 PM To: Mu2e/Offline ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [Mu2e/Offline] Robust random seed plan for digitizers (Issue #849) Initalizing CLHEP HepJamesRandom (setSeed method) 1M times implies one intialization takes about 26e-6s. It involves several times 10k arithmetic operations. A ceSimReco event takes 0.2s, so intializing 20 random engines every event would add 0.25% to the CPU time. — Reply to this email directly, view it on GitHub<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Mu2e_Offline_issues_849-23issuecomment-2D1282916012&d=DwMCaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=Zi5IH-k2_JiK_Wipiv3pwwSXO9UNaHiLP8sisfPz__k&m=HlOYtWl2DtgIKH3_BJpzEDr5JShLajXkt-0ZGAnLQJ_7UfTFKOXeX-R2MwgkcE3G&s=pEttH4kR18Q04W8jbWl3K6StQjbQe68A07OZZXYk4CE&e=>, or unsubscribe<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ABHY5ZQPNJR24GDUNVJ6UIDWD34SPANCNFSM6AAAAAAQK7TFBU&d=DwMCaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=Zi5IH-k2_JiK_Wipiv3pwwSXO9UNaHiLP8sisfPz__k&m=HlOYtWl2DtgIKH3_BJpzEDr5JShLajXkt-0ZGAnLQJ_7UfTFKOXeX-R2MwgkcE3G&s=jlPW1qV_m_6dqxXz1Bhuaeli8pLBW0qwCii2aWI1PiA&e=>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[rlcee]

in units of microseconds per call
HepJamesRandom setSeed(long) = 26
MixMaxRng setSeed(long) = ~0
MixMaxRng setSeeds(long*,1) = 26
MixMaxRng setSeeds(long*,2) = 51
MixMaxRng setSeeds(long*,3) = 75
MixMaxRng setSeeds(long*,4) = 101

[gaponenko]

The scaling of MinMaxRng seed setting is surprisingly efficient. (For those who remember RANLUX.) Andrei

…

On Tue, 25 Oct 2022, Ray Culbertson wrote: in units of microseconds per call HepJamesRandom setSeed(long) = 26 MixMaxRng setSeed(long) = ~0 MixMaxRng setSeeds(long*,1) = 26 MixMaxRng setSeeds(long*,2) = 51 MixMaxRng setSeeds(long*,3) = 75 MixMaxRng setSeeds(long*,4) = 101 -- Reply to this email directly or view it on GitHub: https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Mu2e_Offline_issues_849-23issuecomment-2D1290858010&d=DwICaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=O47fc5vzDTR2V_gla4Ub0Q&m=99t00RuffurC5oh46xDo7wVoAxhMdRv2_F6bqZOZdbW1Ob-4FuJB8kbqwRTmwwVL&s=A0yl7TrM0STXgQ1KZbaGii8jsn4Riu8dR2xHX0Q9cZ8&e= You are receiving this because you are subscribed to this thread. Message ID: ***@***.***>