Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improving PixelVertexCollectionTrimmer #33091

Merged
merged 5 commits into from
Mar 11, 2021

Conversation

missirol
Copy link
Contributor

@missirol missirol commented Mar 7, 2021

PR description:

This PR is meant to improve the implementation of the plugin PixelVertexCollectionTrimmer.

This plugin is used at HLT to select a subset of pixel vertices, based on their Sum-Pt2 score.

In its current form, the plugin implicitly assumes that the input collection of vertices is already sorted according to the same sorting criterion used in PixelVertexCollectionTrimmer itself (the 1st entry of the input collection is assumed to have the highest f.o.m.).

The main update in this PR is to remove this assumption on the ordering of the inputs.

While said assumption was valid in the past (e.g. Run-2 HLT), it might not be valid in the future; for example, the pixel vertices in the Patatrack workflow use a sorting criterion which is slightly different from the "legacy" pixel vertices (the trimmer still uses the same sorting criterion as the legacy pixel vertices).

More details can be found in cms-patatrack#605.

As suggested by @AdrianoDee (TRK POG), I'm opening a PR here, so that experts can review it.


On a technical level, these updates would introduce two small changes in the producer:

  • in the PR, the output collection is sorted according to the sorter in PixelVertexCollectionTrimmer (currently, the trimming is applied, but there is no re-sorting, and the ordering of the outputs is the same as the inputs);

  • the current impl uses sumpt2 > minSumPt2_; in the impl in the PR, this is effectively changed to sumpt2 >= minSumPt2_.


Here are 3 (possibly alternative) ways to maybe improve things further, for experts to consider:

  • adding a data member to reco::Vertex to save the Sum-Pt2 score, such that it would not have to be recomputed a posteriori in other modules (having to make sure different modules use the same sorting criterion);

  • generalizing the implementation of PVClusterComparer (the legacy sorter, used in the trimmer) such that it can be configured to give the legacy sorting, or the Patatrack sorting, depending on what is needed;

  • making PixelVertexCollectionTrimmer capable of using different sorting functions, rather than just PVClusterComparer.


PR validation:

Private tests to verify that the changes work as intended.

Running the updated plugin on 100 events (Run-3 DY MC with PU) suggests that its timing remains very small (<0.1ms).

If this PR is a backport, please specify the original PR and why you need to backport that PR:

N/A (no backport planned)

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 7, 2021

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33091/21416

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 7, 2021

A new Pull Request was created by @missirol (Marino Missiroli) for master.

It involves the following packages:

RecoPixelVertexing/PixelVertexFinding

@perrotta, @jpata, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@makortel, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @ebrondol, @mtosi, @dgulhan this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

@slava77
Copy link
Contributor

slava77 commented Mar 7, 2021

* adding a data member to `reco::Vertex` to save the Sum-Pt2 score, such that it would not have to be recomputed a posteriori in other modules (having to make sure different modules use the same sorting criterion);

sumpt2 is not a particularly unique choice to sort vertices. So, it's less practical to save it as a data member (especially impractical if the computation is done after the vertex is reconstructed). I suggest to save an aligned vector (ValueMap) of sumpt2. This will follow roughly what is done for the offline PV reco.

@slava77
Copy link
Contributor

slava77 commented Mar 7, 2021

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 7, 2021

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-408cb9/13318/summary.html
COMMIT: d88ea6c
CMSSW: CMSSW_11_3_X_2021-03-07-0000/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33091/13318/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-408cb9/34634.0_TTbar_14TeV+2026D76+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-408cb9/34834.999_TTbar_14TeV+2026D76PU_PMXS1S2PR+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+HARVESTGlobalPU

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 3 differences found in the comparisons
  • DQMHistoTests: Total files compared: 38
  • DQMHistoTests: Total histograms compared: 2849195
  • DQMHistoTests: Total failures: 1354
  • DQMHistoTests: Total nulls: 1
  • DQMHistoTests: Total successes: 2847818
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.004 KiB( 37 files compared)
  • DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
  • Checked 160 log files, 37 edm output root files, 38 DQM output files

@missirol
Copy link
Contributor Author

missirol commented Mar 8, 2021

  • the current impl uses sumpt2 > minSumPt2_; in the impl in the PR, this is effectively changed to sumpt2 >= minSumPt2_.

This change caused a large number of (small) differences in the HLT-related comparisons, so I'm reverting it.

PS. Testing with the GRun HLT menu, the differences seems to originate from modules like hltIterL3FromL1MuonTrimmedPixelVertices and hltIterL3MuonTrimmedPixelVertices, which appear to be empty after reverting this last change (and pre-PR).

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 8, 2021

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33091/21440

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 8, 2021

Pull request #33091 was updated. @perrotta, @jpata, @cmsbuild, @slava77 can you please check and sign again.

@perrotta
Copy link
Contributor

The differences in the comparisons are most likely not coming from this PR (the second-to-last comparisons showed no changes in reco/hlt outputs, and the last commit was just a change to a warning message). Is there a way to understand what happened in this last set of comparisons?

@missirol the isssues with wf 11634.911 are known, and as you say do not depend on this PR, see #32963

@missirol
Copy link
Contributor Author

Thanks for the info, @perrotta.

else if (fractionSumPt2_ > 1)
edm::LogWarning("Configuration") << "value of \"fractionSumPt2\" is larger than 1.";

auto const PVcomparerPSet = iConfig.getParameter<edm::ParameterSet>("PVcomparer");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
auto const PVcomparerPSet = iConfig.getParameter<edm::ParameterSet>("PVcomparer");
auto const& pvComparerPSet = iConfig.getParameterSet("PVcomparer");
  • capitalization
  • using a method returning by ref may be cheaper

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment on lines 40 to 43
if (fractionSumPt2_ < 0)
edm::LogWarning("Configuration") << "value of \"fractionSumPt2\" is negative.";
else if (fractionSumPt2_ > 1)
edm::LogWarning("Configuration") << "value of \"fractionSumPt2\" is larger than 1.";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this check is needed, perhaps it's more appropriate to throw?
While a value below 0 does nothing (does it even need a warning in this case?), the value above 1 will by construction make an empty vertex producer and seems more of a reason to think that this is a broken config.

IIUC, considering that this is a stream module this warning will appear nStreams times.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented as suggested (no strong opinion on the choice of warnings or exceptions here).

Comment on lines 52 to 54
edm::LogWarning("Configuration") << "PVcomparer.track_pt_min (" << track_pt_min << ") >= PVcomparer.track_pt_max ("
<< track_pt_max << ") : PVClusterComparer will use pT=" << track_pt_max
<< " for all selected tracks.";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't it make sense to throw here as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment on lines 70 to 71
for (size_t idx = 0; idx < vtxs.size(); ++idx)
foms.at(idx) = pvComparer_->pTSquaredSum(vtxs.at(idx));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for (size_t idx = 0; idx < vtxs.size(); ++idx)
foms.at(idx) = pvComparer_->pTSquaredSum(vtxs.at(idx));
for (size_t idx = 0; idx < vtxs.size(); ++idx)
foms[idx] = pvComparer_->pTSquaredSum(vtxs[idx]);

the index range is correct by construction, no need to check with .at

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

std::iota(sortIdxs.begin(), sortIdxs.end(), 0);
std::sort(sortIdxs.begin(), sortIdxs.end(), [&](size_t const i1, size_t const i2) { return foms[i1] > foms[i2]; });

auto const minFOM_fromFrac = foms.at(sortIdxs.at(0)) * fractionSumPt2_;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
auto const minFOM_fromFrac = foms.at(sortIdxs.at(0)) * fractionSumPt2_;
auto const minFOM_fromFrac = foms[sortIdxs.front()] * fractionSumPt2_;

it looks like range checking is not needed here either

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


auto const minFOM_fromFrac = foms.at(sortIdxs.at(0)) * fractionSumPt2_;

vtxs_trim->reserve(maxVtx_ < vtxs.size() ? maxVtx_ : vtxs.size());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
vtxs_trim->reserve(maxVtx_ < vtxs.size() ? maxVtx_ : vtxs.size());
vtxs_trim->reserve(std::min(maxVtx_, vtxs.size()));

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was my first thought, but it leads to a compiler error

error: no matching function for call to 'min(const uint&, std::vector<reco::Vertex>::size_type)'

I've implemented the suggested solution, but including a type cast.

Comment on lines 83 to 84
if (foms.at(idx) >= minFOM_fromFrac and foms.at(idx) > minSumPt2_)
vtxs_trim->emplace_back(vtxs.at(idx));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (foms.at(idx) >= minFOM_fromFrac and foms.at(idx) > minSumPt2_)
vtxs_trim->emplace_back(vtxs.at(idx));
if (foms[idx] >= minFOM_fromFrac and foms[idx] > minSumPt2_)
vtxs_trim->emplace_back(vtxs[idx]);

seems safe in this context

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


if (vtxs->empty()) {
if (vtxs.empty())
edm::LogWarning("Input") << "Input collection of vertices is empty. Output collection will be empty.";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
edm::LogWarning("Input") << "Input collection of vertices is empty. Output collection will be empty.";
edm::LogWarning("PixelVertexInput") << "Input collection of vertices is empty. Output collection will be empty.";

categories are global and "Input" is a bit too generic if one were to make use of the category without also knowing the module name issuing this warning.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

}

if (vtxs_trim->empty())
edm::LogInfo("Output") << "Output collection is empty.";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
edm::LogInfo("Output") << "Output collection is empty.";
edm::LogInfo("PixelVertexOutput") << "Output collection is empty.";

similar to PixelVertexInput above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@missirol
Copy link
Contributor Author

Thanks for your comments, @slava77. I have tried to implement the suggested changes.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33091/21507

@cmsbuild
Copy link
Contributor

Pull request #33091 was updated. @perrotta, @jpata, @cmsbuild, @slava77 can you please check and sign again.

@slava77
Copy link
Contributor

slava77 commented Mar 10, 2021

@cmsbuild please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-408cb9/13436/summary.html
COMMIT: df509f6
CMSSW: CMSSW_11_3_X_2021-03-10-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33091/13436/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 38
  • DQMHistoTests: Total histograms compared: 2849195
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2849172
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 37 files compared)
  • Checked 160 log files, 37 edm output root files, 38 DQM output files

@slava77
Copy link
Contributor

slava77 commented Mar 11, 2021

+1

for #33091 df509f6

  • code changes are in line with the PR description and the follow up review
    • the updates are technical
    • the PixelVertexCollectionTrimmer is presently used only in HLT
  • jenkins tests pass and comparisons with the baseline show no differences

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

@qliphy
Copy link
Contributor

qliphy commented Mar 11, 2021

+1

@cmsbuild cmsbuild merged commit bf18a63 into cms-sw:master Mar 11, 2021
@missirol missirol deleted the devel_1130pre4_pixelVertexTrimmer branch March 11, 2021 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants