Recovering short tracks continuing saga: adding overlapping hits + outlier rejection

[kmcdermo]

As discussed extensively on the group chat, we are proposing a three step plan for our algorithm to improve efficiency of short-tracks (on top of later tunings of layer window settings, chi2, etc).

The proposal is the following:

1. Forward propagation with one hit per layer as we are already doing with mkFit.
2. Backward propagation to search for extra hits, but do not update when extra hits are added, as suggested originally by @cerati and @areinsvo.
3. Perform our own final fit (forward, backward, zig-zag, etc) to perform outlier rejection + obtain final track parameters as bumped by @IHateLinus .

For starters, we will rely on CMSSW to give us the final fit with outlier rejection, so we should focus on implementing 2. As @mmasciov pointed out, this means that the mtv-like-val in standalone validation will still be sub-optimal, but hopefully improved. We can always run MTV on CMSSW side with our tracks after the final fit in CMSSW to see how we do. However, the hope is that by adding some extra hits even without outlier rejection, we raised our shared hits fraction to help recover some efficiency for short tracks.

To implement 2 properly, we will perhaps need to extend the current max length of the hit array from 32 to something greater so as not to overwrite the last hit index in the array which defeats the whole purpose of appending the list (although now that we stopped appending -1s an nauseam, probably okay...).

We would then have to adapt the current backward fit in MkBuilder to have a window search in between the propagate and update step. This would require that the search is only over hits that are on the opposing overlapping section, so the hits would need an extra bit to say which side of the overlap they are on.

For 3., we can think on the technical implementation. We could still do this in MkBuilder, or pass the completed built tracks to MkFitter (although all the issues of fitting with different nhits / cand will need to be re-addressed). Perhaps it is better to just re-use the final candidates again from the backward propagation to do the final, final fit with outlier rejection.

In any case, in order to remain vectorized, can perform the propagation + update regardless of hit goodness, but choose to store the update based on an evaluation goodness of hit on each layer in outlier rejection (and if it is a hit to be rejected, replace it in the hit list with a -4 or something and simply store the previous updated parameters).

This is tied to Issues #195, #193, #196, and #71 (although more indirectly).

[mmasciov]

Investigating on the short-track inefficiency, I have tracked built track candidates that are found and matched by CMSSW, and looked for those tracks in MkFit. Aim: identify potential

For N(layers)=4:

• found by MkFit: https://mmasciov.web.cern.ch/mmasciov/ineffShortTracks_logs/foundTracks_4lyrs_fullinfo.txt
• lost by MkFit: https://mmasciov.web.cern.ch/mmasciov/ineffShortTracks_logs/lostTracks_4lyrs_fullinfo.txt

For N(layers)=10:

• found by MkFit: https://mmasciov.web.cern.ch/mmasciov/ineffShortTracks_logs/foundTracks_10lyrs_fullinfo.txt
• lost by MkFit: https://mmasciov.web.cern.ch/mmasciov/ineffShortTracks_logs/lostTracks_10lyrs_fullinfo.txt

Few observations:

• MkFit certainly tends to add hits
• Few clear patterns exist for lost tracks (e.g., consecutive missing hits): general enough? Maybe not.
• For certain layers, navigation in MkFit appears to be incorrect (especially looking at longer candidates), or at least NOT the same as CMSSW (and SIM): that is, a layer is 'explored' before another, wrt. CMSSW (and SIM).
• There are significant differences wrt. SIM and CMSSW in the azimuthal angle (φ), also for tracks that are found (!): probably related to the addition of spurious hits. Maybe a hint that our hit selection windows are too generous?

[kmcdermo]

Following the discussion in today's meeting, we have a new proposal for how we want to attack duplicate hits. We want to hear from @cerati and @makortel on this new proposal:

New proposal:

1. During the forward propagation we add the best hit for each layer as we have been doing with update and all. In addition, since we have the hits already there, we store the second best hit from a different module than the first in another array, but do not do any prop+update with the second hit yet (@kmcdermo and @osschar ).
2. Perform the backward propagation (prop+update) with full list of hits from the outward propagation, potentially including the secondary hits from different modules.
3. While in the backward propagation over the seed hits, perform a full hit window search for additional hits from overlapping regions in the pixel layers (with select hit indices, chi2, prop+update).
4. Perform one final fit, testing for outliers.

In order to distinguish hits in different modules, we will need a change in the data format of the Hit class, adding a member that encodes the module number. This needs to happen for both the pixels and strips. Will require a refresh on the binary files. Some discussion on re-ordering stereo layers... can happen simultaneously or sequentially.

One possible addendum to this proposal is to perform outlier rejection by storing the chi2 of each hit added to track or r-phi/r-z distance from the track, and do a cheap rejection after 1. and 3.

In the meanwhile, we are making a few plots to check whether the overlapping pixels or strips is more important for efficiency. We will eventually need volunteers to tackle the segments of this proposal, as they are mostly orthogonal.

[slava77]

I was thinking if we can figure out if the hit is on another module without the detailed layer info and the format update.

• For the barrel mono hits (and [all?] pixel hits) on overlapping modules I'd expect r *delta(phi)/delta(r) << 1.
• For the endcap strips a similar condition (replacing delta(r) with delta(z) should hold.
• The stereo strips are a bit more complicated, I'm thinking that checking the hit position uncertainty direction can be done (strips are not parallel on overlapping modules).

[cerati]

First of all I still think that it's worth exploring ways to increase the efficiency by avoiding adding spurious hits. For instance avoiding two consecutive '-1' hits or a sequence of alternate '>0' and '-1' hits.

The proposed plan for overlapping hits sounds very ambitious, so I would like we figure out if all steps are needed and if we can stage the development. So here are my thoughts:

• Point 3. is the highest priority in my opinion (pixels are the place with most overlaps, and they matter for seeds in other iterations). Except that I would not do the update (I do not see what is the gain). If I remember correctly CMSSW attempts to rebuild the seed region only if a sufficient number of hits is added after the seed (I would do the same).
• Point 1. makes sense to me, but maybe it is not needed if the approach for 3. can work also for strips (i.e. pickup also overlapping strip hits during backward fit). Of course 3. makes more sense for inner strip layers and not much for outer ones, but I think it's more likely to have overlaps in the inner layers (and inner layers are anyways more important)
• I am not sure I understand 2.: what do you mean with "potentially including the secondary hits from different modules"? Do you want to make a decision during the backward fit (e.g. based on the chi2), or you mean that if an overlap hit was identified during fwd you add it? I definitely would not do the former (does not sound very vectorizable), but in any case 2. has to come after 1., so it can make sense only if we decide to do 1.
• I am worried about increasing the Hit object size. Maybe instead of the full detid we can store a couple of bits that tell us the 'sublayer' (based on r or z, along the lines of what Slava is saying)
• If we change the Hit object, I would also add a few bits to store how many strips were in the cluster (we may add a rudimentary cluster shape cut)... but this is a different topic

The final fit is complicated topic. CMSSW does many things in this step (outlier rejection based on chi2, outlier rejection based on pixel templates, evaluation of residuals for alignment, ...). I do not think we want to replace it. We may convince people that we can make a good enough estimation of the track parameters at the interaction point for most of the tracks, so that the slow CMSSW final fit becomes useful only for a subset of tracks (muons, bjets, alignment). In order to do the chi2-based outlier rejection CMSSW has to store the fwd and bwd track states at each hit, and I think this would kill us. We could come up with poor man approaches, like combining somehow the fwd and bwd chi2 (statisticians may not like it, but that's fine as long as we make it work...).

[kmcdermo]

@cerati : allow to me to add some clarification and some context to the discussion today

Point 3. is the highest priority in my opinion (pixels are the place with most overlaps, and they matter for seeds in other iterations). Except that I would not do the update (I do not see what is the gain). If I remember correctly CMSSW attempts to rebuild the seed region only if a sufficient number of hits is added after the seed (I would do the same).

Sure, we don't necessarily need to do the update. We even don't even need to trigger a seed region rebuilding, if we see that we recover enough efficiency. The idea for doing the update was that it helps narrow the search for more duplicates in the next layers of the pixels.

Point 1. makes sense to me, but maybe it is not needed if the approach for 3. can work also for strips (i.e. pickup also overlapping strip hits during backward fit). Of course 3. makes more sense for inner strip layers and not much for outer ones, but I think it's more likely to have overlaps in the inner layers (and inner layers are anyways more important)

So the pickup of strips in backwards fit is actually the original proposal in this issue. However, as @osschar was pointing out, we already have the select hit indices + chi2 testing ready to go in the forward propagation, and we would be effectively relaunching the track finding on the backward fit if we did the pickup for strips on the bwd fit, where there are many more strip layers than pixel layers (so would be quite expensive).

As @mmasciov has been showing, there is plenty of duplicate hits in very long tracks (i.e. those that reach the outer layers), and removing them from CMSSW tracks lowers CMSSW's efficiency. So there is a need to add overlaps in the outer layers it would seem.

I am not sure I understand 2.: what do you mean with "potentially including the secondary hits from different modules"? Do you want to make a decision during the backward fit (e.g. based on the chi2), or you mean that if an overlap hit was identified during fwd you add it?

We mean that we can decide if we want to basically merge the secondary hits into the main hit array, and perform the prop+update with them. May not be needed, but can be an option (since we are proposing to NOT do the prop+update with secondary overlap hits in the forward propagation).

I am worried about increasing the Hit object size. Maybe instead of the full detid we can store a couple of bits that tell us the 'sublayer' (based on r or z, along the lines of what Slava is saying)

Yes, this is similar to what I originally proposed as well, basically a single bit that is written out in the binary file that determines an "inner" and "outer" overlap. @osschar thinks we may be able to use a short or some shortened int type to encode a module number (by removing bits that encode layer number).

[kmcdermo]

re: final fit+outliers @cerati

My understanding from @makortel was that if we give CMSSW our tracks after the building, the fitting in CMSSW handles the outlier rejection for us. Are you saying this not actually the case because we are not passing the fwd+bwd hit chi2s? I guess a simple test is just to see for mkFit tracks if nHits before CMSSW fitting vs after CMSSW fitting is different.

I think we are proposing a poor-man's outlier rejection with a final fit our side because we know how well fitting scales.

[cerati]

I have not seen the evidence for the impact of strip duplicate hits on the efficiency (I know @mmasciov has been posting stuff on the skype chat but I was not able to figure out how it relates to this).

The CMSSW final fit/outlier rejection happens regardless of what we do in mkFit. I am talking about ways to have the outlier rejection in a final fit within mkFit - my main point is that it is probably doable but it is not straightforward to do since our fitter stores significantly less information than the CMSSW one, so we cannot replicate what CMSSW does. I agree it would be nice that we take care also of the fit (and make it faster), but if a CMSSW final fit will stay for other reasons (alignment, proper handling of dead modules, etc) then we should not waste time working on our own version.

[makortel]

CMSSW does many things in this step (outlier rejection based on chi2, outlier rejection based on pixel templates, evaluation of residuals for alignment, ...)

One addition to the list is the pixel cluster position estimation based on the templates (in offline reco, HLT uses generic CPE).

but if a CMSSW final fit will stay for other reasons (alignment,

Why should we specifically care about alignment?

proper handling of dead modules

I don't remember on the top of my head how the final fit deals with dead modules (i.e. whether dead modules are dealt with solely in track building or in final fit, or in both), so it should be checked.

[cerati]

My point is that replacing the CMSSW final fit is something we should definitely consider but before investing time working on it we need to know which outputs we need to produce (and if we can/want to produce those outputs). These outputs are not just the track parameters at the beamline.

Another example is propagation to calorimeter in PF, does it use the outermost state from the track final fit?

By the way you are probably right that the dead modules are probably identified just in the building.