Clarification on the centerline reference for Lane Keeping computation

[EllingtonKirby]

Hi there,

Currently the Lane Keeping score appears to be one of the larger upper bounds on stage two evaluations as seen in the leaderboard.

When looking at the supplementary material, the LK score is described as (emphasis added by me):

The LK score checks whether the ego-vehicle follows the centerline of its current lane

However, in the score computation for lane keeping, it appears to me that the centerline is derived from the PDM-Closed observation via the metric cache:

centerline = self._centerline.linestring

Is centerline derived from the PDM-Closed agent always consistent with the "centerline of the current lane" for the ego vehicle?

I don't have any reason to suspect it isn't, but given that the synthetic scenes appear to sometimes force the ego vehicle off its original lane, I wonder if the PDM Closed trajectory could take a significantly different route from the proposed trajectory, while neither would be strictly "on lane" given the ambiguity of some synthetic scenes.

Although maybe the weighting of synthetic scenes by their distance from the true route handles this discrepancy and the current methods are just truly bad at keeping to their lane under novel scenes 😅 .

Either way thank you again for your contribution to the community with this robust benchmark.