Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataformats and inputs only for PF HEM 15-16 mitigation #24320

Merged
merged 12 commits into from Aug 21, 2018

Conversation

gpetruc
Copy link
Contributor

@gpetruc gpetruc commented Aug 17, 2018

Minimal subset of #24212 with the DataFormat changes and not touching the PF algorithm:

  • add invalidHcal flag to electrons and photons, and fill it in the respective producers
  • add enum for the invalid hcal flag in CaloCluster

This should require no changes to HLT config and have no conflicts anywhere, and so integrating this in an IB should speed up the integration of the rest of the PR with the changes to the PF code.

Still based on CMSSW_10_3_X_2018-08-15-1100

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @gpetruc (Giovanni Petrucciani) for master.

It involves the following packages:

DataFormats/CaloRecHit
DataFormats/EgammaCandidates
RecoEgamma/EgammaElectronAlgos
RecoEgamma/EgammaElectronProducers
RecoEgamma/EgammaIsolationAlgos
RecoEgamma/EgammaPhotonProducers

@perrotta, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@jainshilpi, @Sam-Harper, @varuns23, @rovere, @lgray this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

@gpetruc
Copy link
Contributor Author

gpetruc commented Aug 17, 2018

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 17, 2018

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/29893/console Started: 2018/08/17 11:30

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-24320/29893/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 31
  • DQMHistoTests: Total histograms compared: 2983602
  • DQMHistoTests: Total failures: 2
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2983410
  • DQMHistoTests: Total skipped: 190
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 30 files compared)
  • Checked 129 log files, 14 edm output root files, 31 DQM output files

@gpetruc
Copy link
Contributor Author

gpetruc commented Aug 20, 2018

any chance of getting this one merged?

@slava77
Copy link
Contributor

slava77 commented Aug 21, 2018

+1

for #24320 ee64537

  • code changes are in line with the PR description
  • jenkins tests pass and comparisons with the baseline show no differences
  • local tests on ~1K events in 2018B data gave no events with electrons/photons with invalid hcal. MC tests showed a few events with invalid HCAL.
    • in MC the phase1_2018_realistic 103X_upgrade2018_realistic_v2 GT has HcalChannelQuality with tag HcalChannelQuality_2018_v4.0_mc with about 180 channels masked with status 32270 ( HcalCellTrigMask=15 + HcalCellMask=1).

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

@slava77
Copy link
Contributor

slava77 commented Aug 21, 2018

@fabiocos
it would be nice if this can make it in the -2300 IB.
Thank you.

@abdoulline
Copy link

abdoulline commented Aug 21, 2018

I've played a bit around this (detId and conditions) protection
https://github.com/cms-sw/cmssw/pull/24320/files#diff-9c61d9003eec5179550bfe62d0ea6fbeR128

re-running ("critical") step3 for two wf's from runTheMatrix (2017 and 2013) :
(1) runTheMatrix.py -l 136.388 &
(2) runTheMatrix.py -l 20034.0 &


And while it's good that only good cells are accepted, but arrival of unphysical HcalDetIds in (2):
DEBUG: hasActiveHcal called with 1 detids. First tower detid ieta 0 iphi
EgammaHadTower DetId 43f00000 hid.rawId 43f00000 sub 1 ieta 0 iphi 0 depth 15
(indeed this is HCAL HB with abnormal depth 15 and empty fields for ieta/iphi)
http://cmslxr.fnal.gov/source/DataFormats/HcalDetId/interface/HcalDetId.h#0025)
is quite worrisome. There is no any such HcalDetId in the list of cells printed from CalibCalorimetry/HcalPlugins/src/HcalHardcodeCalibrations.cc
and it (naturally) doesn't appear in HCAL Reco
(in HcalDigis input dumped for the event of interest from HBHEPhase1Reconstructor).


As to (1) - I've added HcalTopology to EgammaHadTower constructor (+ relevant #include's )
edm::ESHandle geoHandle;
es.get().get(geoHandle);
edm::ESHandle hcalTopology;
es.get().get( hcalTopology );
hcalTopology_ = hcalTopology.product();

and then used Sunanda's suggestion (should be effective only for 2017 "collapsed" depth
(and here it's used without intermediate "safety" pointer, but the latter is needed anyway for 2023 wf's) :
int status = hcalQuality_->getValues((DetId)(hcalTopology_->idFront(HcalDetId(id))),false)->getValue();

NB: fixes the issue for this wf. (for 2017 "Plan 1" in general), so it's worth adding it.

@gpetruc @bsunanda
it's painful to compile this PR + 10_3_0_pre2 (160+ packages overnight in my case) but once it's merged, we might need to investigate (using step3 of the mentioned wf's) what's going on with EgammaHadTower (id's) in 2023 case...

@gpetruc I saw a warning "PROPAGATION TO THE HCAL ENTRANCE HAS FAILED" before (2)
crashed in EgammaHadTower. I assume this is a harmless warning (?).

@bsunanda
Copy link
Contributor

bsunanda commented Aug 21, 2018 via email

@abdoulline
Copy link

@bsunanda
I'll send you a recipe (and modified files).

@kpedro88
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit 2cf9223 into cms-sw:master Aug 21, 2018
@kpedro88
Copy link
Contributor

also @abdoulline @bsunanda regarding the HCAL DetId issues: I turned on debug output in https://github.com/cms-sw/cmssw/blob/master/Geometry/CaloTopology/src/CaloTowerConstituentsMap.cc to see if anything jumped out.

In the 2023 case, these messages about a "Tower (0,0)" appear much after all the other towers are constructed, and even after other modules have run. So I think some module is making a bad call to constituentsOf(), or some other strange issue.

Tower (0,0) Depth 0:-1 (HB 0,0,15)
Tower (0,0) Depth 1:0 (HB 0,0,0)
Tower (0,0) Depth 0:4 (HO 0,0)
Tower (0,0) Depth 0:-1 (HB 0,0,15)
Tower (0,0) Depth 1:0 (HB 0,0,0)
Tower (0,0) Depth 0:4 (HO 0,0)
Tower (0,0) Depth 0:-1 (HB 0,0,15)
Tower (0,0) Depth 1:0 (HB 0,0,0)
Tower (0,0) Depth 0:4 (HO 0,0)
Tower (0,0) Depth 0:-1 (HB 0,0,15)
Tower (0,0) Depth 1:0 (HB 0,0,0)
Tower (0,0) Depth 0:4 (HO 0,0)

For 2017, the merged depth DetIds are used by the CaloTower code:

HcalDetId hid = m_hcaltopo->mergedDepthDetId(HcalDetId(HcalEndcap,hcal_ieta*id.zside(),id.iphi(),i+sd));

but the conditions correspond to the unmerged hits, so there will have to be some patch for this.

@bsunanda
Copy link
Contributor

bsunanda commented Aug 21, 2018 via email

@slava77
Copy link
Contributor

slava77 commented Aug 21, 2018

it's merged, but it missed the -2300 IB. So, it should show up in the -1100 IB tomorrow.

@gpetruc gpetruc deleted the PF_103X_noAlgo branch March 8, 2021 09:08
@gpetruc gpetruc restored the PF_103X_noAlgo branch March 8, 2021 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants