Support reroutes in the ground truth

[shankari]

If data collection spans multiple months, then public transit routes can be rerouted. We encountered this mainly on the Berkeley trip. We originally dealt with this by having two different specs, but then there was a third reroute. Simply cloning multiple specs makes it hard to compare across settings.

So we want to combine the specs and fold the various reroutes into the same spec.
Let's document the issues encountered during the merge here.

[shankari]

First, we decided that we will store the various options with validity ranges in the same field.
So the ground truth will look like:

                    "end_loc": [
                    {
                      "type": "Feature",
                      "properties": {
                        "name": "Transbay Terminal"
                        "valid_start_fmt_date": "2019-07-16",
                        "valid_end_fmt_date": "2019-08-30",
                      },
                      "geometry": {
                        "type": "Polygon",
                        "coordinates": [
                          [
                            ...
                          ]
                        ]
                      }
                    },{
                      "type": "Feature",
                      "properties": {
                        "name": "Salesforce Transit Center",
                        "valid_start_fmt_date": "2019-09-01",
                        "valid_end_fmt_date": "2020-04-30",
                      },
                      "geometry": {
                        "type": "Polygon",
                        "coordinates": [
                          [
                            ...
                          ]
                        ]
                      }
                    }
                    ],

[shankari]

An unresolved question about route specs in general and this case in particular, deals with how to represent the trajectory. In general, we have multiple ways of representing the trajectory - you can specify waypoints for OSRM routing, or a OSM relation or a polyline.

The problem with using anything other than a polyline is that the routing can change over time. For example, consider the attached files that represent the relation trajectory for the #F express bus from Emeryville to San Francisco. The first was generated in September 2019. The second was generated in Apr 2020. There are some additional points and some points that have moved slightly.

And if we look at one of the new points (e.g. https://www.openstreetmap.org/#map=19/37.8304134/-122.2831002), it seems to belong to node https://www.openstreetmap.org/node/6935972912. The node was added as part of the stop_area relations for Emery Go-Round in changeset https://www.openstreetmap.org/changeset/76469330 in Oct 2019

[shankari]

this is why we decided to check in the filled specs as well (MobilityNet/mobilitynet-analysis-scripts@5d4f81f).

Given this, should the original specs have their original description or the polylines? If they have polylines, then isn't that redundant to the full coordinates list? Why not just use polylines all through?

Since I don't have anybody to come to a consensus on for now, I will just make the decision and hope to modify it later if there are issues. The current decision is that I will continue to store the relations in the regular specs and store the generated values in the filled specs. This should give us a better idea of how the specs were originally set out while still being able to recreate the trajectories from the filled values.

existing_coords.gz
new_coords.gz

[shankari]

I'm still really torn about this. I am still thinking that it would be useful to be able to run a script to generate the filled value from the spec. But then we would have to find the correct version of OSM to run against and I am not sure how we would deal with OSRM.

And I'm also super paranoid that I will lose the trajectory somehow by running the fill code and then we will never get it back. Which is stupid because everything is checked into github and I can just use the history.

One option is to have both the relation and the polyline as backup. we can always remove the polyline later. I'm going to go with this option even though I can think of many reasons why it is overkill just to get over my dithering.

[shankari]

if we have just a polyline, we don't have any structure to put valid values in. So we are going to add a trajectory field which will have the various trajectory specification methods along with an optional polyline as well.

[shankari]

Actually, with the trajectory field, it is complicated to have multiple sub-objects with the same key. So we just use relation or waypoint_coords with a list instead.

[shankari]

• Change the spec format accepted by the spec filling code (spec_creation/autofill_eval_spec.py)
• Ensure that the filled in spec has the time-dependent ground truth as well
• Change the code that lookup up the ground truth for the evaluation (in emeval/input/spec_details.py - e.g. get_ground_truth_for_leg)

[shankari]

I think we have largely handled this. Closing this so that we can open new issues as needed.