Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

decouple EventContent_cff load from general purpose use of Merge.py #22938

Merged

Conversation

slava77
Copy link
Contributor

@slava77 slava77 commented Apr 11, 2018

This should resolve #22933
The immediate problem is for Configuration/DataProcessing/python/Scenario.py, which defines the T0 config driver for production workflows:

  • the Merge.py is needed for file merging functionality of the Scenario.py
  • the EventContent_cff has to be also loaded for RECO or similar workflows and is normally supposed to be loaded only after the cms.Process object with a relevant era is created (done in the ConfigBuilder.py)
  • In the current state the EventContent_cff is loaded in Merge.py for RECO and similar workflows before cms.Process object is defined. This is a bug.

This fix simply restricts loading of the EventContent_cff in the Merge.py only in merging context for NANO AOD purposes.

Tests:
python $CMSSW_BASE/src/Configuration/DataProcessing/test/RunMerge.py --input-files file:a.root --output-file file:b.root --mergeNANO
and then config dump to see that the result is as expected. The RunMerge is also made aware of mergeNANO.

python $CMSSW_BASE/src/Configuration/DataProcessing/test/RunPromptReco.py --scenario ppEra_Run2_2017_pp_on_XeXe --reco --aod --miniaod --dqmio --global-tag 92X_dataRun2_Express_v7 --lfn=file:raw.root
and then dump to see that _hiCentrality name actually shows up in the keep statements

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @slava77 (Slava Krutelyov) for master.

It involves the following packages:

Configuration/DataProcessing

@cmsbuild, @franzoni, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@Martin-Grunewald this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

@slava77
Copy link
Contributor Author

slava77 commented Apr 11, 2018

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 11, 2018

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/27447/console Started: 2018/04/11 22:37

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22938/27447/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 29
  • DQMHistoTests: Total histograms compared: 2505375
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2505198
  • DQMHistoTests: Total skipped: 176
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 1.06 KiB( 23 files compared)
  • Checked 119 log files, 9 edm output root files, 29 DQM output files

@fabiocos
Copy link
Contributor

@slava77 indeed repeating the test of #22933 with this fix, the CTPPS and GEM event contents, that are added through modifiers, are now present in the regular outputCommands where they are expected to be.

@slava77
Copy link
Contributor Author

slava77 commented Apr 12, 2018

@fabiocos
thank you for checking on your side.
I guess we are good to go with this and #22937

@fabiocos
Copy link
Contributor

@slava77 yes, the Express configuration produced now is identical to the one with the version obtained removing the update to Merge.py that caused the problem. I do not see other possible validations without putting this code in production.

@fabiocos
Copy link
Contributor

+operations

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

@fabiocos
Copy link
Contributor

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Checking Express event content in Configuration/DataProcessing
4 participants