Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HF noise event filter #31328

Merged
merged 4 commits into from
Sep 10, 2020
Merged

HF noise event filter #31328

merged 4 commits into from
Sep 10, 2020

Conversation

lathomas
Copy link
Contributor

@lathomas lathomas commented Sep 2, 2020

PR description:

This PR creates an EDFilter in order to reject events containing a high Et HF hit identified as noise by one of the HCAL flags.
The list of HCAL flags to consider is configurable and currently only contains flags with very small mistag rates.
This PR complements (but doesn't replace) #31271, which aims at building discriminating variables for noise that cannot be easily distinguished from physics signal.
The PR adds one boolean to MINIAOD and NANOAOD.
The motivation for such a filter and the filter performance are described in:
https://indico.cern.ch/event/944298/#sc-10-4-hf-noise-study-on-aod
(in particular slides 4-6)

PR validation:

runTheMatrix gave no error. It was explicitly checked that the new module is run during the MINIAOD step and leads to sensible results.

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 2, 2020

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 2, 2020

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31328/18088

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 2, 2020

A new Pull Request was created by @lathomas for master.

It involves the following packages:

PhysicsTools/NanoAOD
PhysicsTools/PatAlgos
RecoMET/METFilters

@perrotta, @gouskos, @cmsbuild, @fgolf, @slava77, @jpata, @mariadalfonso, @santocch, @peruzzim can you please review it and eventually sign? Thanks.
@jdamgov, @rappoccio, @gouskos, @jdolen, @ahinzmann, @smoortga, @riga, @schoef, @emilbols, @mariadalfonso, @JyothsnaKomaragiri, @nhanvtran, @gkasieczka, @clelange, @hatakeyamak, @ferencek, @gpetruc, @andrzejnovak, @peruzzim, @seemasharmafnal, @mmarionncern this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

@jpata
Copy link
Contributor

jpata commented Sep 2, 2020

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 2, 2020

The tests are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 2, 2020

+1
Tested at: 5f5efa4
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-41be10/9057/summary.html
CMSSW: CMSSW_11_2_X_2020-09-01-2300
SCRAM_ARCH: slc7_amd64_gcc820

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 2, 2020

Comparison job queued.

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 2, 2020

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-41be10/9057/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 156 differences found in the comparisons
  • DQMHistoTests: Total files compared: 35
  • DQMHistoTests: Total histograms compared: 2609667
  • DQMHistoTests: Total failures: 4
  • DQMHistoTests: Total nulls: 9
  • DQMHistoTests: Total successes: 2609632
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.22 KiB( 34 files compared)
  • DQMHistoSizes: changed ( 10024.0,... ): 0.004 KiB MessageLogger/Errors
  • DQMHistoSizes: changed ( 10024.0,... ): 0.004 KiB MessageLogger/Warnings
  • DQMHistoSizes: changed ( 10224.0 ): 0.008 KiB MessageLogger/Warnings
  • DQMHistoSizes: changed ( 1325.7 ): 0.056 KiB Physics/NanoAODDQM
  • Checked 149 log files, 22 edm output root files, 35 DQM output files

Comment on lines 81 to 83
ESHandle<CaloGeometry> pG;
iSetup.get<CaloGeometryRecord>().get(pG);
const CaloGeometry* geo = pG.product();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines 85 to 86
edm::Handle<HFRecHitCollection> theHFhits;
iEvent.getByToken(hfhits_token_, theHFhits);
Copy link
Contributor

@slava77 slava77 Sep 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the is typically used for the data members, better not make it possibly confused with a local.
Also, the handle itself seems unnecessary, get the hit collection directly

Suggested change
edm::Handle<HFRecHitCollection> theHFhits;
iEvent.getByToken(hfhits_token_, theHFhits);
auto const& hfHits = iEvent.get(hfhits_token_);

std::vector<int> noiseBits = getNoiseBits();

//Loop over the HF rechits. If one of them has Et>X and fires one the noise bits, declare the event as bad
for (HFRecHitCollection::const_iterator hfhit = theHFhits->begin(); hfhit != theHFhits->end(); hfhit++) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for (HFRecHitCollection::const_iterator hfhit = theHFhits->begin(); hfhit != theHFhits->end(); hfhit++) {
for (auto const& hfhit : theHFhits) {

range loops are preferred

Comment on lines 101 to 102
for (unsigned int i = 0; i < noiseBits.size(); i++) {
if (noiseBits[i] < 0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for (unsigned int i = 0; i < noiseBits.size(); i++) {
if (noiseBits[i] < 0)
for (auto noiseBit : noiseBits) {
if (noiseBit < 0)

for (unsigned int i = 0; i < noiseBits.size(); i++) {
if (noiseBits[i] < 0)
continue;
if (((hitFlags >> noiseBits[i]) & 1) > 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (((hitFlags >> noiseBits[i]) & 1) > 0) {
if ((hitFlags >> noiseBits[i]) & 1) {

this is apparently enough

Comment on lines 118 to 119
for (unsigned int i = 0; i < listOfNoises_.size(); i++) {
if (listOfNoises_[i] == "HFLongShort")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for (unsigned int i = 0; i < listOfNoises_.size(); i++) {
if (listOfNoises_[i] == "HFLongShort")
for (auto const& noise : listOfNoises_) {
if (noise == "HFLongShort")

}

std::vector<int> HFNoisyHitsFilter::getNoiseBits() const {
std::vector<int> result;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the conversion from listOfNoises_ as a string to a vector of ints (or better yet, HcalPhase1FlagLabels)
should be done in the constructor.
to keep that int/HcalPhase1FlagLabels vector as a const, it may be useful to keep this method as a static member taking const vector<string>& as an argument, it can then be used in the constructor initializer list as noiseBits_(getNoiseBits(iConfig.getParameter<std::vector<std::string>>("listOfNoises")))

Comment on lines 129 to 131
else if (debug_)
edm::LogWarning("HFNoisyHitsFilter")
<< "Couldn't find the bit index associated to this string: " << listOfNoises_[i];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks like a case for an exception, because a wrong noise name string would be a bug in configuration

// ------------ method fills 'descriptions' with the allowed parameters for the module ------------
void HFNoisyHitsFilter::fillDescriptions(edm::ConfigurationDescriptions& descriptions) {
edm::ParameterSetDescription desc;
desc.add<edm::InputTag>("hfrechits", edm::InputTag("reducedHcalRecHits:hfreco"));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
desc.add<edm::InputTag>("hfrechits", edm::InputTag("reducedHcalRecHits:hfreco"));
desc.add<edm::InputTag>("hfrechits", {"reducedHcalRecHits:hfreco"});

minor, but still

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 7, 2020

+1
Tested at: 298be1f
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-41be10/9160/summary.html
CMSSW: CMSSW_11_2_X_2020-09-06-2300
SCRAM_ARCH: slc7_amd64_gcc820

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 7, 2020

Comparison job queued.

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 7, 2020

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-41be10/9160/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 156 differences found in the comparisons
  • DQMHistoTests: Total files compared: 35
  • DQMHistoTests: Total histograms compared: 2609667
  • DQMHistoTests: Total failures: 4
  • DQMHistoTests: Total nulls: 9
  • DQMHistoTests: Total successes: 2609632
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.22 KiB( 34 files compared)
  • DQMHistoSizes: changed ( 10024.0,... ): 0.004 KiB MessageLogger/Errors
  • DQMHistoSizes: changed ( 10024.0,... ): 0.004 KiB MessageLogger/Warnings
  • DQMHistoSizes: changed ( 10224.0 ): 0.008 KiB MessageLogger/Warnings
  • DQMHistoSizes: changed ( 1325.7 ): 0.056 KiB Physics/NanoAODDQM
  • Checked 149 log files, 22 edm output root files, 35 DQM output files

@mariadalfonso
Copy link
Contributor

@lathomas

#31375
basically propagate the same HCAL severity level downstream.
Do you need both ? what is the interplay ?

@lathomas
Copy link
Contributor Author

lathomas commented Sep 7, 2020

@mariadalfonso Yes we need both
#31375 adds the noisy HF rechits to MINIAOD. This PR introduces the MET filter that could then be run on MINIAOD once #31375 is merged.

@slava77
Copy link
Contributor

slava77 commented Sep 8, 2020

+1

for #31328 298be1f

  • code changes are in line with the PR description and the follow up review
  • jenkins tests pass and comparisons with the baseline show differences only in the TriggerResults, which corresponds to a new MET path and an update to the combined MET path (the diff script can not cross-correlate old path indices, so an insertion of a new path looks like many changes)

@mariadalfonso
Copy link
Contributor

+xpog
a new filter is added in the miniAOD met filters, propagated to the nanoAOD as well

@santocch
Copy link

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

@silviodonato
Copy link
Contributor

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants