Assertion failure in TrackCollectionCloner::Producer::operator() #37018
Comments
|
assign reconstruction |
|
FYI @cms-sw/tracking-pog-l2 |
|
New categories assigned: reconstruction @slava77,@jpata,@clacaputo you have been requested to review this Pull request/Issue and eventually sign? Thanks |
|
A new Issue was created by @makortel Matti Kortelainen. @Dr15Jones, @perrotta, @dpiparo, @makortel, @smuzaffar, @qliphy can you please review it and eventually sign/assign? Thanks. cms-bot commands are listed here |
|
I guess this was caused by #36793 (because I can't imagine how the mkFit PR could have caused it) |
|
adding some more circumstantial evidence, the assertion crash doesn't happen at the first track nor at the first event processed in wf 23234.9, but rather in the second event. adding: - assert(selTrajs_->back().measurements().size() == trk.recHitsSize());
+ std::cout << selTrajs_->back().measurements().size() << " ==== " << trk.recHitsSize() << std::endl;
+ assert(selTrajs_->back().measurements().size() == trk.recHitsSize()); one gets: also the fact that the only wf affected concerns vector hits seems to be pointing to something done explicitly for that case. |
|
also adding @JanFSchulte here (for the vectorhits-related discussion) |
|
Indeed 23234.9 looks to be the only vectorHits-enabled test of the IB. And simply removing that process modifier prevents the failure. It is interesting the comment in the code just above it https://github.com/cms-sw/cmssw/blob/master/RecoTracker/FinalTrackSelectors/src/TrackCollectionCloner.cc#L47 : "// we assume tracks and trajectories are one-to-one and the assocMap is useless" . Well, that "useless" map is exactly what I am using in #36793 see https://github.com/cms-sw/cmssw/blob/master/RecoMTD/TrackExtender/plugins/TrackExtenderWithMTD.cc#L724 |
|
assign upgrade |
|
New categories assigned: upgrade @AdrianoDee,@srimanob you have been requested to review this Pull request/Issue and eventually sign? Thanks |
|
It is my understanding that VectorHits is an ongoing R&D effort and not yet converged. |
The VectorHit reconstruction is needed to pursue seeding in the Outer Tracker, which needless to say is a must for Phase-2. |
|
Adding some more anedoctal evidence the track that triggers the assertion is the first one without any Inner Tracker hit: |
|
I have started to look into this now and can confirm that it only happens for tracks reconstructed by the pixelLess iteration that is added with the vectorHits modifier. Since the other tracking iterations are still using the vectorHits, the conclusion is that the hits themselves are fine and the issue is introduced in the code specific to the new iteration. IIRC the only piece of code that is unique to that iteration is the seed generator https://github.com/cms-sw/cmssw/blob/master/RecoTracker/TkSeedGenerator/plugins/SeedingOTEDProducer.cc, so I will start digging there. |
|
indeed excluding diff --git a/RecoTracker/FinalTrackSelectors/python/earlyGeneralTracks_cfi.py b/RecoTracker/FinalTrackSelectors/python/earlyGeneralTracks_cfi.py
index c33f6cbfd71..5b8d9de60fc 100644
--- a/RecoTracker/FinalTrackSelectors/python/earlyGeneralTracks_cfi.py
+++ b/RecoTracker/FinalTrackSelectors/python/earlyGeneralTracks_cfi.py
@@ -114,5 +114,5 @@ def _extend_pixelLess(x):
x.indivShareFrac += [0.095]
x.selectedTrackQuals += ['pixelLessStepSelector:pixelLessStep']
x.setsToMerge[0].tLists += [6]
-(trackingPhase2PU140 & vectorHits).toModify(earlyGeneralTracks, _extend_pixelLess)
+#(trackingPhase2PU140 & vectorHits).toModify(earlyGeneralTracks, _extend_pixelLess)
diff --git a/RecoTracker/IterativeTracking/python/InitialStep_cff.py b/RecoTracker/IterativeTracking/python/InitialStep_cff.py
index 2bf9f026faf..f70d6ed189c 100644
--- a/RecoTracker/IterativeTracking/python/InitialStep_cff.py
+++ b/RecoTracker/IterativeTracking/python/InitialStep_cff.py
@@ -424,7 +424,9 @@ trackingPhase2PU140.toModify(initialStepSelector,
# Final sequence
-InitialStepTask = cms.Task(initialStepSeedLayers,
+from RecoLocalTracker.SiPhase2VectorHitBuilder.siPhase2VectorHits_cfi import *
+InitialStepTask = cms.Task(siPhase2VectorHits,
+ initialStepSeedLayers,
initialStepTrackingRegions,
initialStepHitDoublets,
initialStepHitTriplets,
diff --git a/RecoTracker/IterativeTracking/python/iterativeTkConfig.py b/RecoTracker/IterativeTracking/python/iterativeTkConfig.py
index 9790ee16d26..868ca252184 100644
--- a/RecoTracker/IterativeTracking/python/iterativeTkConfig.py
+++ b/RecoTracker/IterativeTracking/python/iterativeTkConfig.py
@@ -61,8 +61,8 @@ _iterations_trackingPhase2PU140 = [
"DetachedQuadStep",
"PixelPairStep",
]
-from Configuration.ProcessModifiers.vectorHits_cff import vectorHits
-vectorHits.toModify(_iterations_trackingPhase2PU140, func=lambda x: x.append('PixelLessStep'))
+#from Configuration.ProcessModifiers.vectorHits_cff import vectorHits
+#vectorHits.toModify(_iterations_trackingPhase2PU140, func=lambda x: x.append('PixelLessStep'))makes 23234.9 step 3 run to completion (of course defying the whole purpose of VH) |
|
@JanFSchulte @cms-sw/tracking-pog-l2 note that according to the discussion at the ORP today, this is blocking for the upcoming release next week. The other options are to revert the MTD TrackExtender PR, or to disable the vectorhits workflow to be addressed in a follow-up. |
why is that? Is that a technical shortcoming of the infrastructure or just policy.
neither of the two seem wise IMHO. |
|
@mmusich this issue is crashing wf 23234.9 in the IB. |
|
This seems to be originated from the fact that the A simple solution would be to add: --- a/RecoTracker/IterativeTracking/python/PixelLessStep_cff.py
+++ b/RecoTracker/IterativeTracking/python/PixelLessStep_cff.py
@@ -355,6 +355,8 @@ pixelLessStepTracks = RecoTracker.TrackProducer.TrackProducer_cfi.TrackProducer.
)
fastSim.toModify(pixelLessStepTracks, TTRHBuilder = 'WithoutRefit')
+from Configuration.Eras.Modifier_phase2_timing_layer_cff import phase2_timing_layer
+phase2_timing_layer.toModify(pixelLessStepTracks, TrajectoryInEvent = True)
# TRACK SELECTION AND QUALITY FLAG SETTING.which restore the alignment of the collections (and makes the comment valid ). |
thanks for this. Indeed this escaped my attention. |
|
@mmusich @AdrianoDee in my PR I focussed only on the collections that appeared to be needed for the |
|
+upgrade |
In CMSSW_12_3_X_2022-02-21-1100 workflow 23234.9 step 3 fails with
https://cmssdt.cern.ch/SDT/cgi-bin/logreader/slc7_amd64_gcc10/CMSSW_12_3_X_2022-02-21-1100/pyRelValMatrixLogs/run/23234.9_TTbar_14TeV+2026D49_vectorHits+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTrigger+RecoGlobal+HARVESTGlobal/step3_TTbar_14TeV+2026D49_vectorHits+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTrigger+RecoGlobal+HARVESTGlobal.log#/
The text was updated successfully, but these errors were encountered: