Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tag-N-Probe] Split read mode #22861

Merged
merged 7 commits into from Apr 20, 2018

Conversation

drkovalskyi
Copy link
Contributor

Added a new mode of running TagNProbe code that uses less memory by creating RooDataSet for each bin individually and reading data from the input TTree in chunks. It allows to reduce memory usage for typical fits done by Muon POG from ~20GB down to 1-2GB. The code runs significantly slower in the new mode, so it shouldn't be a default option. The change should not affect any code unless the new mode is requested explicitly and even in this case all results should be the same.

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 5, 2018

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 5, 2018

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 5, 2018

A new Pull Request was created by @drkovalskyi for master.

It involves the following packages:

PhysicsTools/TagAndProbe

@gpetruc, @cmsbuild, @arizzi, @monttj can you please review it and eventually sign? Thanks.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

@drkovalskyi
Copy link
Contributor Author

@gpetruc, @cmsbuild, @arizzi, @monttj could you please check the pull request?

@gpetruc
Copy link
Contributor

gpetruc commented Apr 18, 2018

At a quick read it looks ok to me, but me and @arizzi are supposed to put the +1 only on NanoAOD-related PRs, and this one is not.
Did you do a check that on a small dataset, for which the old version works, the two things give the same answer, and that it works also correctly for dynamic categories? (the one practical case I'm aware of is the fit for tracking efficency, maybe you can ask Walaa for a cfg or to do the test)

@drkovalskyi
Copy link
Contributor Author

I have checked muon efficiency measures for Run2017 and got identical results, but I cannot test all use cases since I'm not familiar with them and I don't see any unit tests to check it. So far we started using it in Muon POG for new measurements, so it's pretty much production already.

@fabiocos
Copy link
Contributor

@drkovalskyi I understand this is not running in standard production wf, is it correct? Or which are them in case?

@drkovalskyi
Copy link
Contributor Author

It's not running in the standard production, but it's widely used by different groups.

@fabiocos
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 19, 2018

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/27571/console Started: 2018/04/19 16:09

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22861/27571/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 29
  • DQMHistoTests: Total histograms compared: 2492830
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2492653
  • DQMHistoTests: Total skipped: 176
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 23 files compared)
  • Checked 119 log files, 9 edm output root files, 29 DQM output files

@fabiocos
Copy link
Contributor

+1

@fabiocos
Copy link
Contributor

merge

@cmsbuild cmsbuild merged commit 7ecedc2 into cms-sw:master Apr 20, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants