Geant4Output2EDM4hep: get run from worker context by andresailer · Pull Request #1663 · AIDASoft/DD4hep

andresailer · 2026-08-21T15:42:21Z

This will need some more work, ~~the current status writes one run per thread.~~

BEGINRELEASENOTES

Geant4OutputAction: In multithreaded mode, make sure the ingested runparameters and fileparameters end up in the output, fixes OutputAction and contexts #1660
- Fixed locking endRun action of the outputaction
Tests: added comparison of run and metadata trees if they exist in the output

ENDRELEASENOTES

TODO

Output2ROOT
Output2LCIO: writes one run per thread
Common refactor?

github-actions · 2026-08-21T16:57:46Z

Test Results

9 files 9 suites 11h 44m 19s ⏱️
49 tests 34 ✅ 0 💤 15 ❌
410 runs 387 ✅ 0 💤 23 ❌

For more details on these failures, see this check.

Results for commit 1eb7ad4.

♻️ This comment has been updated with latest results.

tmadlener · 2026-08-21T17:21:00Z

the current status writes one run per thread.

I'm assuming a "trivial" solution employing std::call_once doesn't do the trick? Might be a bit trickier if there are multiple runs as well as multiple threads...

tmadlener

This looks deceptively simple in the end, but I think following your comments / explanations it should be correct.

Somewhat related: Can we handle multiple runs in flight in a multi-threaded context correctly in DD4hep/ddsim? I think at least from the podio side things should keep working, as long as all runs are added to a worker context at some point.

andresailer · 2026-08-27T05:49:15Z

Somewhat related: Can we handle multiple runs in flight in a multi-threaded context correctly in DD4hep/ddsim? I think at least from the podio side things should keep working, as long as all runs are added to a worker context at some point.

Unless I already forgot again: run information is read when the file is opened. There is no sequential processing of files. Only one run per file, so no treatment of multiple runs in parallel.

andresailer · 2026-08-27T05:59:58Z

+    // only one of the worker contexts (or the master context) has run parameters, since only one reader is created
+    // which registers the run parameters
+    parameters->extractParameters(runHeader);
+    m_file->writeFrame(runHeader, podio::Category::Run);


I have to check. Are there always runParameters, or are there cases where we don't have runParameters, but we still want to write the runheader.

only write run frame when we have run parameters

andresailer · 2026-08-27T08:32:09Z

deceptively

Indeed

use same printout identifier as during createWorker

andresailer · 2026-08-28T08:15:45Z

EventReader test fails only in 107a
106: no EDM4hepFileReader, podio is too old
107a: issue with

16: Warning in <TClass::Init>: no dictionary for class podio::root_utils::CollectionWriteInfo is available

(podio 1.2.0)

109: this warning is not present (podio 1.7.0)

Still weird why it started failing so, need to investigate.

andresailer · 2026-08-28T08:16:27Z

Still weird why it started failing so, need to investigate.

Because we just turned on EDM4hep for the CI

tmadlener · 2026-08-28T08:17:48Z

What is the file version with which the input file has been written? (podio-dump on the input file should tell you). Since podio doesn't make forward compatibility guarantees, it's possible that the input file is too new for certain stacks.

andresailer · 2026-08-28T08:26:40Z

possible that the input file is too new for certain stacks.

106

podio-dump crashes

107a

TClass::Init:0: RuntimeWarning: no dictionary for class podio::root_utils::CollectionWriteInfo is available
input file: ../DDTest/inputFiles/ZH250_ISR.edm4hep.root
            (written with podio version: 1.6.0)

datamodel model definitions stored in this file: 
 - edm4hep (0.99.99)

Frame categories in this file:
Name    Entries
------  ---------
ERROR: Cannot print category 'events' (not present in file)
$ rootls ../DDTest/inputFiles/ZH250_ISR.edm4hep.root
events  podio_metadata  runs

109a

podio-dump DDTest/inputFiles/ZH250_ISR.edm4hep.root 
input file: DDTest/inputFiles/ZH250_ISR.edm4hep.root
            (written with podio version: 1.6.0)

datamodel model definitions stored in this file:
 - edm4hep (0.99.99)

Frame categories in this file:
Name    Entries  
------  -------  
runs    1        
events  3        
################################### events: 0 ####################################
Collections:
Name (s = subset collection)  ValueType                          Size  ID          
----------------------------  ---------------------------------  ----  --------    
EventHeader                   edm4hep::EventHeader               1     d793ab91    
GeneratorEventParameters      edm4hep::GeneratorEventParameters  1     a193a48b    
MCParticles                   edm4hep::MCParticle                125   a1cba250    

Parameters:
Name  Type  Elements  
----  ----  --------

andresailer · 2026-08-28T08:51:56Z

I am going to disable the edm4hep filereader test for podio < 1.6

…llbacks

…mutex Thanks to D. Konstantinov and his codex

Avoids having the mutex present where it is not needed

… register another callback

…ur file is too new

…ound the action here

…and put them into the file in the end

Thank you gemini ❤️

…in multithreaded mode

andresailer mentioned this pull request Aug 21, 2026

OutputAction and contexts #1660

Open

andresailer force-pushed the run2multi branch from 130bce7 to ddc9c33 Compare August 26, 2026 11:31

andresailer marked this pull request as ready for review August 26, 2026 11:31

andresailer force-pushed the run2multi branch 2 times, most recently from 86c6456 to 2c073a2 Compare August 26, 2026 11:33

andresailer requested a review from tmadlener August 26, 2026 11:33

tmadlener reviewed Aug 26, 2026

View reviewed changes

andresailer force-pushed the run2multi branch from 2c073a2 to f8c5d60 Compare August 27, 2026 05:49

andresailer enabled auto-merge (rebase) August 27, 2026 05:58

andresailer disabled auto-merge August 27, 2026 05:59

andresailer commented Aug 27, 2026

View reviewed changes

Geant4Output2EDM4hep: get run from worker context

28e4672

only write run frame when we have run parameters

andresailer force-pushed the run2multi branch from f8c5d60 to eb8b30c Compare August 27, 2026 07:44

DDTest: silence the physics printouts from geant4 for ddsim tests

9e8ea59

andresailer force-pushed the run2multi branch from eb8b30c to 9e8ea59 Compare August 27, 2026 08:16

andresailer commented Aug 27, 2026

View reviewed changes

Comment thread DDG4/edm4hep/Geant4Output2EDM4hep.cpp Outdated

andresailer added 2 commits August 27, 2026 11:01

Geant4Kernel: fix error message about missing worker

4921e45

use same printout identifier as during createWorker

CI: Enable EDM4hep for all newer stacks

61274fb

andresailer commented Aug 27, 2026

View reviewed changes

Comment thread DDG4/include/DDG4/Geant4Action.h Outdated

andresailer commented Aug 27, 2026

View reviewed changes

Comment thread DDG4/edm4hep/Geant4Output2EDM4hep.cpp Outdated

andresailer force-pushed the run2multi branch from bfdae93 to 01e306c Compare August 28, 2026 08:51

andresailer force-pushed the run2multi branch from 7f9f65a to 085d46b Compare August 28, 2026 12:49

andresailer commented Aug 28, 2026

View reviewed changes

Comment thread DDTest/python/compare_simulation_output.py Outdated

andresailer force-pushed the run2multi branch 4 times, most recently from 8b461b0 to bc2c601 Compare August 28, 2026 13:37

andresailer commented Aug 28, 2026

View reviewed changes

Comment thread DDTest/python/compare_simulation_output.py

andresailer commented Aug 28, 2026

View reviewed changes

Comment thread DDG4/plugins/Geant4EventSeed.cpp

andresailer commented Aug 28, 2026

View reviewed changes

Comment thread DDG4/lcio/Geant4Output2LCIO.cpp Outdated

andresailer added 13 commits August 28, 2026 17:31

Geant4Action: make mutex a member so we can properly lock endOfRun Ca…

695f5e2

…llbacks

Geant4Output2EDM4hep: fix obtaining file parameters from worker context

11465d5

Geant4Kernel: remove unsed m_threadContext member

d66814d

EventActionSequenceAdopt: protect by its own mutex instead of action …

1ee6aae

…mutex Thanks to D. Konstantinov and his codex

Geant4Action: move mutex to EventAction and EventActionSequence

b6aad35

Avoids having the mutex present where it is not needed

EventSeeder: in ddsim this is added to runAction, so we don't need to…

ef93ac1

… register another callback

test_EventReader: only test for podio_version larger 1.6, otherwise o…

5de286e

…ur file is too new

TestMT: only read edm4hep file if podio > 1.6.0

fcd3e1d

EventAction: move lock one call later, since we don't have to lock ar…

dbdc48a

…ound the action here

DDTest: only test edm4hep plugins for podio > 1.6.0

dbb6d00

Geant4Output2EDM4hep: store the run and file parameters from any run …

6e521e3

…and put them into the file in the end

Test: mt_compare: add comparison of metadata and runs trees

4165bc0

Thank you gemini ❤️

Geant4Output2ROOT: replace static mutex with mutex member

19b60e3

andresailer force-pushed the run2multi branch from 01ca473 to 814ace3 Compare August 28, 2026 15:31

Geant4Output2LCIO: use member mutex, fix exception about missing run …

1eb7ad4

…in multithreaded mode

andresailer force-pushed the run2multi branch from 814ace3 to 1eb7ad4 Compare August 28, 2026 16:00

Uh oh!

Conversation

andresailer commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

tmadlener commented Aug 21, 2026

Uh oh!

tmadlener left a comment

Choose a reason for hiding this comment

Uh oh!

andresailer commented Aug 27, 2026

Uh oh!

andresailer Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

andresailer commented Aug 27, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

andresailer commented Aug 28, 2026

Uh oh!

andresailer commented Aug 28, 2026

Uh oh!

tmadlener commented Aug 28, 2026

Uh oh!

andresailer commented Aug 28, 2026

106

107a

109a

Uh oh!

andresailer commented Aug 28, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

andresailer commented Aug 21, 2026 •

edited

Loading

github-actions Bot commented Aug 21, 2026 •

edited

Loading