Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[JMENanoAOD] Backport of #39000 (Updates for Run-3) to 12_4_X #39078

Merged
merged 9 commits into from Aug 22, 2022
Merged

[JMENanoAOD] Backport of #39000 (Updates for Run-3) to 12_4_X #39078

merged 9 commits into from Aug 22, 2022

Conversation

nurfikri89
Copy link
Contributor

Backport from #39000

Orignal PR description:

This PR updates JMENano to be compatible with Run-3 data and MC, starting with the upcoming NanoAODv10 production. The following major changes are made:

  • Sequences have been changed to Tasks and JMENano can now run in Prompt.
  • Updated names of NanoAOD-related config files and modules.
  • Add a dedicated function for reclustering and content customisation of AK4 Puppi jets.
  • Recluster both AK4 CHS and AK4 Puppi jet collections but modify their table names and content depending on era (i.e Run-2 or Run-3).
  • Increase pt threshold for stored reclustered AK8 reco jets to 15 GeV.
  • The JMENano workflows are enabled back again, which was disabled by PR Disable JME Nano workflows due to Nano-prompt dev #35140. Enable Run-3 workflows and added one wf: 11634.15.

@nurfikri89
import pfCHS
1b7029c
@nurfikri89
Switch from Sequence to Task for prompt production
32aed86
@nurfikri89
Update custom_jme_cff.py for Run-3 and NanoV10 compatibility (12_5_X).
f51225d 
- Update name of NanoAOD-related config files and modules
- Add adedicated function for reclustering and content customisation
of AK4 Puppi jets.
- Recluster both AK4 CHS and AK4 Puppi jet collections but modify
their table names and content depending on eras	(i.e Run-2 or Run-3).
@nurfikri89
Re-enable JMENano workflows
4bc1c43
@nurfikri89
Clean up toModify and fix typo
abb51fb
@nurfikri89
Increase pt cut threshold of stored reclustered AK8 reco jets to 15 G…
7e0e062 
…eV. Use >= for condition in Var
@nurfikri89
Add wf for Run-3 JMENano
b54dabc
@nurfikri89
Ensure that PuppiProducer uses existingWeights as stored in Mini
a799dc7
@nurfikri89
Increase precision for JEC rawFactor
c5f6008
@cmsbuild cmsbuild added this to the CMSSW_12_4_X milestone Aug 16, 2022
@cmsbuild
Copy link
Contributor

A new Pull Request was created by @nurfikri89 (Nurfikri Norjoharuddeen) for CMSSW_12_4_X.

It involves the following packages:

  • Configuration/PyReleaseValidation (pdmv, upgrade)
  • PhysicsTools/NanoAOD (xpog)
  • PhysicsTools/PatAlgos (reconstruction)

@gouskos, @swertz, @jordan-martins, @vlimant, @bbilin, @clacaputo, @cmsbuild, @AdrianoDee, @srimanob, @kskovpen, @jpata, @mariadalfonso, @mandrenguyen can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @schoef, @emilbols, @kpedro88, @Martin-Grunewald, @trtomei, @mbluj, @ahinzmann, @demuller, @seemasharmafnal, @mmarionncern, @jdolen, @makortel, @fabiocos, @missirol, @azotz, @slomeo, @beaucero, @jdamgov, @nhanvtran, @gkasieczka, @hatakeyamak, @mariadalfonso, @AlexDeMoor, @swertz, @JyothsnaKomaragiri, @gpetruc, @andrzejnovak this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

@mariadalfonso
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7bf073/26856/summary.html
COMMIT: c5f6008
CMSSW: CMSSW_12_4_X_2022-08-16-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39078/26856/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 8 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3677242
  • DQMHistoTests: Total failures: 13
  • DQMHistoTests: Total nulls: 1
  • DQMHistoTests: Total successes: 3677206
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -0.004 KiB( 49 files compared)
  • DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
  • Checked 208 log files, 45 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

@jpata
Copy link
Contributor

jpata commented Aug 17, 2022 •
edited

+reconstruction

@srimanob
Copy link
Contributor

test parameters:

  • workflows = 10224.15,11024.15,25202.15,11634.15

@srimanob
Copy link
Contributor

@cmsbuild please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7bf073/26893/summary.html
COMMIT: c5f6008
CMSSW: CMSSW_12_4_X_2022-08-17-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39078/26893/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-7bf073/10224.15_TTbar_13+2017PU_JMENano+TTbar_13TeV_TuneCUETP8M1_GenSim+DigiPU+RecoFakeHLTPU+HARVESTFakeHLTPU+Nano
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-7bf073/11024.15_TTbar_13+2018PU_JMENano+TTbar_13TeV_TuneCUETP8M1_GenSim+DigiPU+RecoFakeHLTPU+HARVESTFakeHLTPU+Nano
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-7bf073/11634.15_TTbar_14TeV+2021_JMENano+TTbar_14TeV_TuneCP5_GenSim+Digi+RecoNano+HARVESTNano+ALCA
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-7bf073/25202.15_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25+NANOUP15MC_PU25_JME

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3677242
  • DQMHistoTests: Total failures: 2
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3677218
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 208 log files, 45 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

@mariadalfonso
Copy link
Contributor

+xpog
backport inline with master

remaining items notes here in master's PR
#39000 (comment)

@srimanob
Copy link
Contributor

+Upgrade

Backport PR

@kskovpen
Copy link
Contributor

+pdmv

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_4_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_5_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

@perrotta
Copy link
Contributor

@mariadalfonso your comment in #39000 (comment) implied that this PR cannot be set to central production until items 2 and 5 from
#39000 (comment) aren't solved.

Does your signature in this backport PR (which, if merged, will automatically include this PR in all future central nanoAOD productions in 12_4_X) means that these issues are now to be considered to be solved? Can we safely merge this in 12_4_X since now, then?

@mariadalfonso
Copy link
Contributor

@perrotta
This PRs does not interfere with the main nanoV10 analysis nano and only affect the custom JMEnano. Those are effectively two different productions that can use different releases (i.e. 12_4_6 for analysis nano and 12_4_7 for JMEnano)

We should merge this PR now as it is, as it can help the private production of JMEnano:
JME is producing them now with them aim of deriving residuals, and this can serve as first validation.

@perrotta
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit f3a9a2f into cms-sw:CMSSW_12_4_X Aug 22, 2022
@nurfikri89 nurfikri89 deleted the portFrom125XTo124X_JMENanoRun3_forPR branch October 29, 2023 06:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
fully-signed orp-approved pdmv-approved reconstruction-approved tests-approved upgrade-approved xpog-approved
Projects
None yet
Milestone
CMSSW_12_4_X
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@nurfikri89 @cmsbuild @mariadalfonso @jpata @srimanob @kskovpen @perrotta