Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properly handle Run product merging in LHESource #21820

Merged
merged 8 commits into from Jan 12, 2018

Conversation

Dr15Jones
Copy link
Contributor

The LHESource was not properly handling the merging of the LHERunInfoProduct when multiple files are read. The framework requires that the source explicilty tell the framework a new file is being opened which setup the system for proper merging.

This change also fixed a crash in the LHESource which happened if multiple files were read.

Previously, when the first xml document was finished and the XMLDocument
object was deleted, then xerces would be uninitialized. This lead to
the parser being invalid so the next read of LHEReader would crash.
Keeping the XercesPlatform alive for the lifetime of the LHEReader
fixes that problem.
Moved to std::unique_ptr and std::shared_ptr for most memory handling.
Changed member variables to match CMS conventions.
When we identify that we need to merge the LHERunInfoProduct in
nextEvent, we immediately do the merge.
This avoids the need for a endRun call which will allow those calls
to be removed from the framework.
The function fileIndex was not being overridden for base class
ProducerSourceBase. Now we forward FromFile::fileIndex to
ProducerSourceBase.
By telling the framework that the LHESource has gone to a new file,
the merging of RunInfoProduct can be done properly.
@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 9, 2018

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 9, 2018

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 9, 2018

A new Pull Request was created by @Dr15Jones (Chris Jones) for master.

It involves the following packages:

FWCore/Sources
GeneratorInterface/LHEInterface

@cmsbuild, @smuzaffar, @efeyazgan, @Dr15Jones, @perrozzi can you please review it and eventually sign? Thanks.
@mkirsano, @makortel, @wddgit, @Martin-Grunewald, @agrohsje, @alberto-sanchez this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

@Dr15Jones
Copy link
Contributor Author

please test

@Dr15Jones
Copy link
Contributor Author

+1

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 9, 2018

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/25350/console Started: 2018/01/09 21:27

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 9, 2018

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 9, 2018

Comparison job queued.

@@ -0,0 +1,153 @@
Number of particles = 12
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi, is this log file needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is used by the unit test. The test compares the output of the test job to this log file and if there is a difference, the test fails.

@@ -0,0 +1,197 @@
Number of particles = 12
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also used by the unit test.

@perrozzi
Copy link
Contributor

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

@Dr15Jones
Copy link
Contributor Author

Fixes issue #21807

@fabiocos
Copy link
Contributor

please test workflow 513.0

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 12, 2018

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/25397/console Started: 2018/01/12 09:36

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21820/25397/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

  • /build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-21820/1325.7_TTbar_13_94XNanoAODINPUT+TTbar_13_94XNanoAODINPUT+NANOEDMMC2017
  • /build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-21820/513.0_WTolNuJets_LO_Mad_13TeV_py8+WTolNu01234Jets_5f_LO_MLM_Madgraph_LHE_13TeV+Hadronizer_TuneCUETP8M1_13TeV_MLM_5f_max4j_LHE_pythia8+HARVESTGEN2

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 6 differences found in the comparisons
  • DQMHistoTests: Total files compared: 26
  • DQMHistoTests: Total histograms compared: 2467710
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2467540
  • DQMHistoTests: Total skipped: 169
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 1.06999999993 KiB( 22 files compared)
  • Checked 110 log files, 9 edm output root files, 26 DQM output files

@fabiocos
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit 03e7453 into cms-sw:master Jan 12, 2018
@Dr15Jones Dr15Jones deleted the lheSourceFixes branch January 20, 2018 01:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants