Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix hot HCAL channels in legacy 2016 #22713

Merged
merged 19 commits into from May 25, 2018

Conversation

deguio
Copy link
Contributor

@deguio deguio commented Mar 22, 2018

this PR is supposed to provide a recipe to fix the hot channels problem affecting HE and HF in the 2016 legacy rereco. the issue is described at:
https://baylor.app.box.com/s/n0yrtd0p2grzngv33jx0qtdae0tvwu8e

the hot cells are being cured at PFCandidate level and can be applied at the miniAODv2 step by adding
--customise_unsch PhysicsTools/PatAlgos/slimming/customizeMiniAOD_HcalFixLegacy2016.custo\ mizeAll

while the HE hot cells are cured, some tuning is still needed in HF.
@gpetruc

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 22, 2018

A new Pull Request was created by @deguio (Federico De Guio) for CMSSW_9_4_X.

It involves the following packages:

CommonTools/ParticleFlow
PhysicsTools/PatAlgos

@perrotta, @monttj, @cmsbuild, @slava77, @gpetruc, @arizzi can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @imarches, @ahinzmann, @acaudron, @gpetruc, @mmarionncern, @jdamgov, @jdolen, @nhanvtran, @JyothsnaKomaragiri, @gkasieczka, @seemasharmafnal, @schoef, @ferencek, @mverzett, @mariadalfonso, @pvmulder, @TaiSakuma, @cbernet this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

#include "DataFormats/Common/interface/Association.h"
#include <iostream>

class PFCandidateRecalibrator : public edm::EDProducer {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(new) legacy modules are not allowed.
Please use stream:: or global::

process.load("RecoVertex.AdaptiveVertexFinder.inclusiveVertexing_cff")
task.add(process.inclusiveVertexingTask)
task.add(process.inclusiveCandidateVertexingTask)
task.add(process.inclusiveCandidateVertexingCvsLTask)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this list looks kind of incomplete.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for this I let @gpetruc comment

@slava77
Copy link
Contributor

slava77 commented Mar 22, 2018

@deguio

There should be a PR for the master branch first.
Please make one and add a note in this PR about it.

complete integration of this will have to be done on top of CMSSW_9_4_MAOD_X branch.
Please change the target branch of this PR (it should be possible from the web interface).

IIUC, this is needed for the re-miniAOD based on 2016 80X legacy outputs.
Proper testing will need to have this proposed customization enabled in the standard jenkins tests (e.g. 136.7611 or 1325.5).
Current preference is to have all necessary code enabled in the workflow with process modifiers (not with customise methods).
So, your PR will need to make use of run2_miniAOD_80XLegacy "era".
If era solution becomes impossible, the customise will need to be added to the matrix workflows (also to become a part of this PR)

@deguio deguio changed the base branch from CMSSW_9_4_X to CMSSW_9_4_MAOD_X March 22, 2018 16:49
@slava77
Copy link
Contributor

slava77 commented Mar 23, 2018

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 23, 2018

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/27085/console Started: 2018/03/23 23:10

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22713/27085/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 27
  • DQMHistoTests: Total histograms compared: 2721493
  • DQMHistoTests: Total failures: 101
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2721230
  • DQMHistoTests: Total skipped: 162
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0 KiB( 0 files compared)
  • Checked 110 log files, 9 edm output root files, 27 DQM output files

@slava77
Copy link
Contributor

slava77 commented Mar 26, 2018

@deguio
what is the status of the PR for the master branch?

@deguio
Copy link
Contributor Author

deguio commented Mar 26, 2018

hi @slava77
sorry I didn't have time to implement your suggestions yet. I will do that soon and open a PR to master as well

edm::ESHandle<CaloGeometry> calogeom;
iSetup.get<CaloGeometryRecord>().get(calogeom);
const CaloGeometry* cgeo = calogeom.product();
HcalGeometry* hgeom = (HcalGeometry*)(cgeo->getSubdetectorGeometry(DetId::Hcal,HcalForward));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it obvious that all of the conditions accessed above will never change during a run?

Also, if a job is processing a skim and there are multiple runs in the same IOV, it seems wasteful to recompute all values.
Better change to use ESWatchers
https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideHowToGetDataFromES#Seeing_that_a_Record_has_changed

void PFCandidateRecalibrator::beginRun(const edm::Run &iRun, const edm::EventSetup &iSetup)
{
//Get Calib Constants from current GT
edm::ESHandle<HcalDbService> GTCond;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

start local variables and data members with lower case letter. Capitalization is reserved for class names.


void PFCandidateRecalibrator::beginRun(const edm::Run &iRun, const edm::EventSetup &iSetup)
{
//Get Calib Constants from current GT
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a reset to badCh vectors.
A job processing multiple runs will have these vectors grow at best redundantly, but at worst it will introduce dependence on the order of processed events if two runs have different sets of bad channels

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is done in the last commit

@cmsbuild
Copy link
Contributor

Pull request #22713 was updated. @perrotta, @prebello, @kpedro88, @fabozzi, @cmsbuild, @gpetruc, @slava77, @GurpreetSinghChahal, @monttj, @arizzi can you please check and sign again.

@fabiocos
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented May 24, 2018

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/28153/console Started: 2018/05/24 11:52

@fabiocos
Copy link
Contributor

fabiocos commented May 24, 2018

@lpernie I have just removed any change to the autoCond.py and relaunched the test, I will take for good the previous signatures of PdmV @fabozzi and reconstruction @slava77 (but their confirmation is welcome)
@kpedro88 could you please check and sign in case?

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22713/28153/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 20 differences found in the comparisons
  • DQMHistoTests: Total files compared: 27
  • DQMHistoTests: Total histograms compared: 2721493
  • DQMHistoTests: Total failures: 142
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2721189
  • DQMHistoTests: Total skipped: 162
  • DQMHistoTests: Total Missing objects: 0

@fabozzi
Copy link
Contributor

fabozzi commented May 24, 2018

+1

@lpernie
Copy link
Contributor

lpernie commented May 24, 2018

+1

1 similar comment
@kpedro88
Copy link
Contributor

+1

@fabiocos
Copy link
Contributor

@slava77 the update didn't concern directly reco code, I assume you confirm your signature

@deguio
Copy link
Contributor Author

deguio commented May 24, 2018

thanks @fabiocos
and sorry for the slow feedback...

@fabiocos
Copy link
Contributor

+1

the reconstruction part was unchanged since the signature 4 days ago, just the GT has been updated.

@fabiocos
Copy link
Contributor

merge

@cmsbuild cmsbuild merged commit 494a276 into cms-sw:CMSSW_9_4_MAOD_X May 25, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

10 participants