implement a tracking particle associator with TrackCandidate and add trackCandidate branches in trackingNtuple

[slava77]

• SimDataFormats/Associations:
  • add alias template to simplify definition of SimToRecoCollection* and RecoToSimCollection* and removed its partial duplicate definition in TrackToTrackingParticleAssociator.h to keep everything just in TrackAssociation.h. The fully defined types should be the same, so no update should be necessary in the classes* files for the dictionaries. There are no use cases at this point for placing SimToRecoCollectionTCandidate and RecoToSimCollectionTCandidate in edm::Event; so, no dictionary is added for this.
  • changed TrackToTrackingParticleAssociator::associateRecoToSim arguments/interface from Handle<TrackingParticleCollection to RefVector<TrackingParticleCollection to be more in common with existing use cases for reco::Tracks
• SimTracker/TrackAssociatorProducers: improve templated methods in QuickTrackAssociatorByHitsImpl so that reco::Track and TrackCandidate have maximal overlap in implementation and minimize copy-pasted code
• Validation/RecoTrack: trackingNtuple updates
  • add tcand_* branches to provide access to TrackCandidate details (optional, enabled for extendedContent custom option). This should allow more straightforward debugging of track pattern recognition, in particular in mkFit. This data can also be used to train track candidate classification and allow removing candidates from the following final fit step (and save computation)
  • refactor mergeIters case (where built tracks are used as inputs) to provide access to per-iteration MVAs
  • add (global) hit positions for rechits as computed with the track-trajectory corrected CPEs to allow for hit position and residual analysis
  • cleanup the outdated use of TransientTrackingRecHitBuilder and use tracking hits directly. This is in a sense a somewhat late follow-up to TTRHs removed from Tracker code for good. #3344

No changes are expected in the standard workflows.
If that is confirmed, there may be a backport to an older release (12_2_X where the TRK-Run3Winter22 are made, in case it's not practical to run on those inputs in 12_4_X)

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37829/29745

ERROR: Build errors found during clang-tidy run.

SimTracker/TrackAssociatorProducers/plugins/QuickTrackAssociatorByHitsImpl.cc:206:26: error: no matching function for call to 'getTrackAt' [clang-diagnostic-error]
    const auto* pTrack = ::getTrackAt(trackCollection, i);
                         ^
SimTracker/TrackAssociatorProducers/plugins/QuickTrackAssociatorByHitsImpl.cc:786:12: note: in instantiation of function template specialization 'QuickTrackAssociatorByHitsImpl::associateRecoToSimImplementation<edm::AssociationMap<edm::OneToManyWithQualityGeneric<std::vector<TrackCandidate>, std::vector<TrackingParticle>, double, unsigned int, edm::RefProd<std::vector<TrackCandidate>>, edm::RefProd<std::vector<TrackingParticle>>, edm::Ref<std::vector<TrackCandidate>, TrackCandidate, edm::refhelper::FindUsingAdvance<std::vector<TrackCandidate>, TrackCandidate>>, edm::Ref<std::vector<TrackingParticle>, TrackingParticle, edm::refhelper::FindUsingAdvance<std::vector<TrackingParticle>, TrackingParticle>>>>, edm::Handle<std::vector<TrackCandidate>>, edm::RefVector<std::vector<TrackingParticle>, TrackingParticle, edm::refhelper::FindUsingAdvance<std::vector<TrackingParticle>, TrackingParticle>>, TrackerHitAssociator>' requested here
--
SimTracker/TrackAssociatorProducers/plugins/QuickTrackAssociatorByHitsImpl.cc:271:26: error: no matching function for call to 'getTrackAt' [clang-diagnostic-error]
    const auto* pTrack = ::getTrackAt(trackCollection, i);
                         ^
SimTracker/TrackAssociatorProducers/plugins/QuickTrackAssociatorByHitsImpl.cc:798:12: note: in instantiation of function template specialization 'QuickTrackAssociatorByHitsImpl::associateSimToRecoImplementation<edm::AssociationMap<edm::OneToManyWithQualityGeneric<std::vector<TrackingParticle>, std::vector<TrackCandidate>, double, unsigned int, edm::RefProd<std::vector<TrackingParticle>>, edm::RefProd<std::vector<TrackCandidate>>, edm::Ref<std::vector<TrackingParticle>, TrackingParticle, edm::refhelper::FindUsingAdvance<std::vector<TrackingParticle>, TrackingParticle>>, edm::Ref<std::vector<TrackCandidate>, TrackCandidate, edm::refhelper::FindUsingAdvance<std::vector<TrackCandidate>, TrackCandidate>>>>, edm::Handle<std::vector<TrackCandidate>>, edm::RefVector<std::vector<TrackingParticle>, TrackingParticle, edm::refhelper::FindUsingAdvance<std::vector<TrackingParticle>, TrackingParticle>>, TrackerHitAssociator>' requested here
--
gmake: *** [config/SCRAM/GMake/Makefile.coderules:129: code-checks] Error 2
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

[slava77]

@smuzaffar @makortel
is scramb code-checks-all now supposed to cover everything including code-checks and code-format? or do I need to run those separately?

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37829/29748

• This PR adds an extra 72KB to repository

[cmsbuild]

A new Pull Request was created by @slava77 (Slava Krutelyov) for master.

It involves the following packages:

• SimDataFormats/Associations (simulation)
• SimTracker/TrackAssociatorProducers (simulation)
• Validation/RecoTrack (dqm)

@civanch, @emanueleusai, @mdhildreth, @cmsbuild, @jfernan2, @ahmad3213, @pmandrik, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@fabiocos, @apsallid, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @bsunanda, @ebrondol, @lecriste, @wmtford, @mtosi, @mmusich, @threus, @dgulhan this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[slava77]

@cmsbuild please test

[smuzaffar]

@slava77 , scram b code-checks runs clang-tidy for changed files while code-checks-all runs clang-tidy on all checkedout files. There are code-format and code-format-all to run the clang-format

[slava77]

scram b code-checks runs clang-tidy for changed files

OK. In the end I have probably just missed the error when I ran code-checks locally.

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-baef6c/24485/summary.html
COMMIT: 1261b6a
CMSSW: CMSSW_12_4_X_2022-05-05-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37829/24485/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 6 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3700704
• DQMHistoTests: Total failures: 14
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3700668
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 206 log files, 45 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[slava77]

• Reco comparison results: 6 differences found in the comparisons
• DQMHistoTests: Total failures: 14

as expected there are no relevant differences (apart for a few false-positives from the warning messages in PU workflows).

[emanueleusai]

+1

[civanch]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1