-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle event meta data changes in streamer files [14_0] #44978
Handle event meta data changes in streamer files [14_0] #44978
Conversation
The event meta-data (e.g. BranchLists) is now added as an EventMsg before the following EventMsg for which it applies. This added EventMsg contains no data products. When seen, the added EventMsg is handled as an artificial file boundary in order to cause the needed stream synchronization. This change is needed to handle how HLT merges parts of streamer files which might have different meta-data.
- event meta data is cached at begin job or when new file is opened - refactored StreamerOutputModuleCommon to be two classes so buffer could be handled separately for the caching - added many unit tests to show GlobalEvFOutputModule properly writes the expected events and the data products are correct
A new Pull Request was created by @Dr15Jones for CMSSW_14_0_X. It involves the following packages:
@cmsbuild, @nothingface0, @smuzaffar, @rvenditti, @tjavaid, @antoniovagnerini, @saumyaphor4252, @Dr15Jones, @syuvivida, @smorovic, @emeschi, @perrotta, @makortel, @consuegs can you please review it and eventually sign? Thanks. cms-bot commands are listed here
|
cms-bot internal usage |
please test |
@Dr15Jones Using 14_0_6, most of the DQM clients (
Is this incompatibility expected? |
Yes. The streamer format had to be changed to accommodate the needed meta data. The streamer format has no backwards compatibility ability (by its original design). You'll have to do what was described before, convert the original streamer into a ROOT file then use the ROOT file as input to a new job and have that new job write the new streamer format. |
-1 Failed Tests: RelVals-INPUT RelVals-INPUT
Comparison SummarySummary:
|
@cmsbuild please test |
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12a253/39404/summary.html Comparison SummarySummary:
|
+core (this is just to have my signature on the record, avoid the future merge to become pending on |
Hi Matti, this PR is on hold. From the e-mail thread, one had the impression the tests were successful, but it would be good to have confirmation. |
backport of #44892 |
+1
|
+alca
|
unhold |
This pull request is fully signed and it will be integrated in one of the next CMSSW_14_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_14_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @sextonkennedy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2) |
+1 |
PR description:
Moved storage of meta data needed for events into the first EventMessage sent before the subsequent events which need that meta data.
This fixed a problem seen online where different HLT nodes were generating different event meta data for the same luminosity block but online the event data for one node is used.
PR validation:
Code compiles and unit tests pass.
backport of #44892 (and #44753 to make the former trivial to backport)