Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hcal depth energy fraction packed candidates value map #26465

Conversation

hatakeyamak
Copy link
Contributor

@hatakeyamak hatakeyamak commented Apr 16, 2019

PR description:

Includes HCAL depth information for packed candidates in MINIAOD for high-level object use (e.g. PUPPI recomputation at the post ultra-legacy MINIAOD stage). The promising results are obtained [1,2], but needs more time to finalize it. It means it is important to have this information in packedCandidates, so that the PUPPI weights can be recomputed on-the-fly from MINIAOD.

[0] https://github.com/cms-sw/cmssw/files/3056535/he_depth_reco.pdf
[1] https://indico.cern.ch/event/799365/contributions/3333038/attachments/1805670/2946660/pre.pdf
[2] https://indico.cern.ch/event/799365/contributions/3339621/attachments/1805694/2946705/JK_JetMET_03_19_2.pdf

If this ValueMap-based implementation as suggested by @slava77 and @peruzzim looks good, this may supersede:
[3] #26363
Now. I also use vectors instead of arrays to store various depth information.

@peruzzim @fgolf @zdemirag @ahinzmann @bendavid @mariadalfonso

PR validation:

Made sure that the this information gets added to MINIAOD as intended. The size increase in TTbar sample with PU (tested in 2000 events) in 2017 & 2018 scenarios is

  • 0.0% as expected for 2017
  • 2.9% for 2018
    a little larger increase than what [3] gives for 2018 (2.7%) but very similar.

if this PR is a backport please specify the original PR:

This is not a backport.

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-26465/9294

  • This PR adds an extra 40KB to repository

@slava77
Copy link
Contributor

slava77 commented Apr 16, 2019

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 16, 2019

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/34218/console Started: 2019/04/16 14:53

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @hatakeyamak (Kenichi Hatakeyama) for master.

It involves the following packages:

PhysicsTools/PatAlgos

@perrotta, @cmsbuild, @santocch, @slava77 can you please review it and eventually sign? Thanks.
@TaiSakuma, @gouskos, @rappoccio, @HeinerTholen, @peruzzim, @seemasharmafnal, @mmarionncern, @ahinzmann, @smoortga, @acaudron, @jdolen, @drkovalskyi, @ferencek, @jdamgov, @nhanvtran, @gkasieczka, @schoef, @clelange, @JyothsnaKomaragiri, @mverzett, @gpetruc, @mariadalfonso, @pvmulder this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

@hatakeyamak
Copy link
Contributor Author

FYI: I used this very simple python file to check this addition.
https://baylor.box.com/s/b3jpnwbi7xvyjs8fof05z7ivuxrbk5d3

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Pull request #26465 was updated. @perrotta, @cmsbuild, @santocch, @slava77 can you please check and sign again.

@slava77
Copy link
Contributor

slava77 commented Apr 22, 2019

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 22, 2019

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/34296/console Started: 2019/04/23 00:33

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26465/34296/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 33
  • DQMHistoTests: Total histograms compared: 3211964
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3211759
  • DQMHistoTests: Total skipped: 204
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 32 files compared)
  • Checked 137 log files, 14 edm output root files, 33 DQM output files

@slava77
Copy link
Contributor

slava77 commented Apr 23, 2019

Here is a plot from JetHT2018C wf 136.876

all_orig-patVSsign1073-pat_RunJetHT2018Cwf136p876c_patHcalDepthEnergyFractionsedmValueMap_packedPFCandidates_hcalDepthEnergyFractions_PAT_obj_values__fraction0

It looks like the setup works OK.

@hatakeyamak
Copy link
Contributor Author

Good. I also checked locally for one data workflow before I made the last commit. So, I guess it looks good to sign off?

source="std::vector<uint8_t> fractionsI_"
target="fractions_">
<![CDATA[ newObj->initFloatVector();
]]>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static analyzer picked up on this line
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26465/34296/llvm-analysis/report-9a8c17.html#EndPath

According to it, the code for this rule looks like the following

 pat::HcalDepthEnergyFractions* newObj = (pat::HcalDepthEnergyFractions*)target;

 if (oldObj) {}

 if (newObj) {}

  newObj->initFloatVector();

@pcanal @Dr15Jones
By some magic (?) I would expect that the body of CDATA[ should end up inside if (newObj) {.
Are we using the syntax for ioread incorrectly or should we simply add if (newObj) { ?

We have the same pattern in a few other ioread rules in CMSSW. So, a more general solution is perhaps outside the scope of this PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That code is there solely to prevent unused variable warning. It needs to be updated to be (void)newObj, etc.

@slava77
Copy link
Contributor

slava77 commented Apr 23, 2019

+1

for #26465 e574dd3

  • code changes are in line with the PR description and the follow up review
  • jenkins tests pass and comparisons with the baseline show differences only in the output products.
    • A simple monitoring of fractions was added to the reco comparisons and should appear in the next update of the cms-bot reco comparison scripts soon.

@fabiocos
Copy link
Contributor

+1

@fabiocos
Copy link
Contributor

merge

@cmsbuild cmsbuild merged commit 669d289 into cms-sw:master Apr 26, 2019
@santocch
Copy link

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants