Test JME-custom NANOAOD production

[fabiocos]

This issue is meant to keep track of the comment #28430 (comment) about the usefulness to add a test workflow (or at least a test configuration that could be used in private tests) to probe the JME=custom NANOAOD variant.

@peruzzim @nurfikri89 FYI

[cmsbuild]

A new Issue was created by @fabiocos Fabio Cossutti.

@davidlange6, @Dr15Jones, @smuzaffar, @fabiocos, @kpedro88 can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[fabiocos]

assign xpog

[cmsbuild]

New categories assigned: xpog

@fgolf,@peruzzim you have been requested to review this Pull request/Issue and eventually sign? Thanks

[peruzzim]

assign pdmv

following #28430 (comment)
@nurfikri89

[cmsbuild]

New categories assigned: pdmv

@chayanit,@pgunnell,@zhenhu you have been requested to review this Pull request/Issue and eventually sign? Thanks

[srimanob]

Example of cmsDriver,
cmsDriver.py myNanoProdMc2016 -s NANO --mc --eventcontent NANOAODSIM --datatier NANOAODSIM --no_exec --conditions 102X_mcRun2_asymptotic_v7 --era Run2_2016,run2_nanoAOD_94X2016 --customise_commands="process.add_(cms.Service('InitRootHandlers', EnableIMT = cms.untracked.bool(False)))\nfrom PhysicsTools.NanoAOD.custom_jme_cff import PrepJMECustomNanoAOD; PrepJMECustomNanoAOD(process)\n"

Full information, at the end of the page
https://twiki.cern.ch/twiki/bin/view/CMS/JMECustomNanoAOD

[srimanob]

I am trying to implement JME Nano to relvals, but I found this issue (*). Any idea what is missing?

The MiniAOD root file and python file can be found in
/afs/cern.ch/user/s/srimanob/public/ForJetMET/NanoJME
(Please ignore part of EDM, it will be flat ntuple when merging happens in central production)
You can check out 11_1_0_pre2, and then run w/o checking anything.
2 pythons are in that directory w/ and w/o JME customization. Python w/o customization ran fine.

The cmsDriver (up to NanoAOD) comes from
https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?prep_id=CMSSW_11_0_0_pre13__fullsim_PU_2016-1575737502-H125GGgluonfusion_13
where NanoAOD ran fine.

One difference is "run2_nanoAOD_94X2016". It should not be used w/ MiniAOD from 11 or 10_6 release.

(*)
----- Begin Fatal Exception 24-Jan-2020 11:14:46 CET-----------------------
An exception of category 'UnknownUserInt' occurred while
[0] Processing Event run: 1 lumi: 1 event: 8 stream: 0
[1] Running path 'nanoAOD_step'
[2] Calling method for module SimpleCandidateFlatTableProducer/'AK4PFPUPPITable'
Exception Message:
Requested UserInt looseId is not available! Possible UserInts are:
tightId tightIdLepVeto
----- End Fatal Exception -------------------------------------------------

[nurfikri89]

Hi @srimanob! Apologies for the silence. I will take a look at this.

[srimanob]

Can the error be duplicated?
Do you need any more information?

FYI @ahinzmann @lathomas

[nurfikri89]

@srimanob I was able to duplicate the error and I am in the process of looking into it. My suspicion is that I have not been very careful with the era modifiers (particularly related to 2016 MC and data production) in the custom_nano_jme.py. Apologies for being slow on this!

[peruzzim]

@nurfikri89 please check the latest version of jets_cff, we have added era modifiers run2_jme_2016 and run2_jme_2017 to be able to separate things linked to 2016/2017 as data taking years (e.g. ID versions) from things that need rerunning only on 2016 80X old MiniAOD samples (run2_miniAOD_80XLegacy)

[nurfikri89]

@peruzzim thanks for the pointers. I will check it out. Out of curiosity, is there a 2018 modifier? There would be some IDs (like pileup jet ID) with different training/configurations for 2018 also.

[peruzzim]

please set the 2018 ones as default and customise for the other years

[nurfikri89]

@peruzzim @srimanob I got it to work now. I did not setup the right modifiers in jetCollectionTools.py. I also realised that I was not scheduling the proper PatJetIDValueMapProducer according to the type of input use for the jet clustering (i.e CHS vs PUPPI). I will push the changes to my branch and then I will make a PR.

[srimanob]

@nurfikri89 Thanks!

@peruzzim @chayanit @pgunnell
Maybe it's better if PR of relvals goes in the same time. I can publish the PR of workflows if PdmV agrees so we can have the one to test with, or JME would like to have it in your PR is also fine too. So this story can be closed.

Basically, we can just have
steps['NANOUP15_PU25_JME']=merge([{'--customise_commands':'"from PhysicsTools.NanoAOD.custom_jme_cff import PrepJMECustomNanoAOD; PrepJMECustomNanoAOD(process)"'},steps['NANOUP15']])
in relvals_step (and maybe clone ones for 17/18) and new workflows to run them.

[chayanit]

@nurfikri89 Thanks!

@peruzzim @chayanit @pgunnell
Maybe it's better if PR of relvals goes in the same time. I can publish the PR of workflows if PdmV agrees so we can have the one to test with, or JME would like to have it in your PR is also fine too. So this story can be closed.

Basically, we can just have
steps['NANOUP15_PU25_JME']=merge([{'--customise_commands':'"from PhysicsTools.NanoAOD.custom_jme_cff import PrepJMECustomNanoAOD; PrepJMECustomNanoAOD(process)"'},steps['NANOUP15']])
in relvals_step (and maybe clone ones for 17/18) and new workflows to run them.

I have no further comment. If you could help with the RelVals PR that would be great, thank you very much @srimanob

[nurfikri89]

@srimanob Can you provide a data relval file which I can test for running on data? At the moment, it is not setup to run on data but I can make the modifications.

[nurfikri89]

I've made the PR (#28854) which should fix the problem @srimanob encountered. It also includes the changes which will enable it to run on Data.

[peruzzim]

+xpog
I think we can close this for the moment

[srimanob]

It seems I missed to work on workflow for data, I will take care of that.
@nurfikri89 @peruzzim

[fabiocos]

@srimanob @peruzzim do I properly understand that this issue can be considered as solved?

[vlimant]

please close