Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix collapsed status in HCAL topology #22001

Merged
merged 5 commits into from Feb 3, 2018

Conversation

kpedro88
Copy link
Contributor

#21842 unintentionally introduced some discrepancies in the 2018 "collapsed" workflow. The underlying problem is that HcalTopology::withSpecialRBXHBHE() is set by the underlying geometry, and since we use a geometry that allows collapsing for 2018, it is always true, even if the collapsing is not actually performed.

Instead, from now on we will use HcalTopology::getMergePositionFlag(), which can be set in Python. This parameter will be set by the same modifier that puts the RecHit combiner module into the HCAL local reco sequence.

This change is propagated to all necessary areas of the code. @abdoulline has confirmed that it works as expected for CaloTowers (with thresholds removed, since sometimes there can be weird threshold effects for collapsed vs uncollapsed RecHits): https://cms-cpt-software.web.cern.ch/cms-cpt-software/General/Validation/SVSuite/HCAL/calo_scan_single_pi/10_0_0/10_0_0_2018C_fixNocuts_vs_10_0_0_2018UC_fixNocuts_SinglePi/

@Martin-Grunewald for HLT, the parameter process.HcalTopologyIdealEP.MergePosition should be false by default, except for collapsed menus (2017 and 2018). What is the best way to specify this in the customization?

This PR will be backported to 100X.

attn: @bsunanda

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@kpedro88
Copy link
Contributor Author

please test

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 28, 2018

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/25702/console Started: 2018/01/28 16:40

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @kpedro88 (Kevin Pedro) for master.

It involves the following packages:

Configuration/PyReleaseValidation
DQMOffline/Hcal
Geometry/CaloTopology
Geometry/HcalEventSetup
Geometry/HcalTowerAlgo
HLTrigger/Configuration
RecoLocalCalo/CaloTowersCreator
RecoLocalCalo/HcalRecAlgos
RecoLocalCalo/HcalRecProducers
RecoParticleFlow/PFClusterProducer

@perrotta, @cmsbuild, @prebello, @Dr15Jones, @vazzolini, @kmaeshima, @civanch, @fwyzard, @ianna, @kpedro88, @fabozzi, @Martin-Grunewald, @silviodonato, @jfernan2, @mdhildreth, @slava77, @GurpreetSinghChahal, @vanbesien, @dmitrijus can you please review it and eventually sign? Thanks.
@ghellwig, @mmarionncern, @makortel, @felicepantaleo, @rovere, @lgray, @Martin-Grunewald, @ebrondol, @seemasharmafnal, @jalimena, @cbernet, @mariadalfonso, @argiro, @geoff-smith, @bachtis, @rociovilar this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22001/25724/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

  • /build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22001/1325.7_TTbar_13_94XNanoAODINPUT+TTbar_13_94XNanoAODINPUT+NANOEDMMC2017

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 1443 differences found in the comparisons
  • DQMHistoTests: Total files compared: 26
  • DQMHistoTests: Total histograms compared: 2464350
  • DQMHistoTests: Total failures: 4299
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2459882
  • DQMHistoTests: Total skipped: 169
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 1.40999999992 KiB( 22 files compared)
  • Checked 110 log files, 9 edm output root files, 26 DQM output files

@ianna
Copy link
Contributor

ianna commented Jan 30, 2018

+1

@kpedro88
Copy link
Contributor Author

+1

@Martin-Grunewald
Copy link
Contributor

+1

@jfernan2
Copy link
Contributor

+1

@perrotta
Copy link
Contributor

perrotta commented Jan 31, 2018

@kpedro88 : while looking at the jenkins tests outputs, I notice significant differences in the standard, "non-collapsed", 10824.0 TTbar 2018 workflow even outside the HE region, e.g.:

image

Can you check and comment on it?

@perrotta
Copy link
Contributor

Moreover: you say in #22001 (comment) that changes are expected because previous PR (was it #21842?) the topology was incorrectly being treated as collapsed. However in #21842 there were not changes in wf 10824.0, that I would have expected if the topology was incorrectly being treated as collapsed there: am I missing something?

@kpedro88
Copy link
Contributor Author

@perrotta the previous PR had some inconsistencies, but the problem actually started when we updated the geometry in the GT to use the collapsible version. The CaloTower plot changes (outside of 28/29) look like random fluctuations to me due to the change in HcalGeometry which could affect GEN-SIM.

@abdoulline
Copy link

abdoulline commented Jan 31, 2018 via email

@abdoulline
Copy link

Sorry for being too fast... Kevin, you're right, there are small SimHits changes visible in the plots linked above...

@perrotta
Copy link
Contributor

perrotta commented Feb 1, 2018

Thank you @kpedro88 and @abdoulline for the explanations.

I would have liked to check in the jenkins tests outputs of #21854 (the PR which supposedly exposed the bug fixed here) the change in shape witnessing the use of the collapsed scenario even for "standard" 2018 workflows. However, they already disappeared.

Therefore, I checked with one of the "collapsed" workflows the same eta distribution above of the CaloTowers, and I can verify that the collapsed distribution
calotowersorted_collapsed
does correspond to what is black in #22001 (comment).

That confirms the issue, and the efficacy of the fix applied here.

@perrotta
Copy link
Contributor

perrotta commented Feb 1, 2018

+1

  • The fix recovers the "non-collapsed" geometry for the "standard" 2018 workflows
  • Changes in jenkins outputs are consistent with that fix and with the random fluctuations due to the change in HcalGeometry which affects SimHits
  • "Collapsed" workflows, not in the standard jenkins tests, were also tested to work

@fabiocos
Copy link
Contributor

fabiocos commented Feb 3, 2018

@prebello you had already signed it the pdmv part of the PR, so I understand that this is ok for you

@fabiocos
Copy link
Contributor

fabiocos commented Feb 3, 2018

merge

@cmsbuild cmsbuild merged commit d1b6abe into cms-sw:master Feb 3, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet