Python code to evaluate the APLS metric
Switch branches/tags
Nothing to show
Clone or download
Latest commit f001ba4 Jan 8, 2018


Python code to evaluate the APLS metric



Code Overview

This code evaluates the Average Path Length Similarity (APLS) metric to measure the difference between ground truth and proposal graphs. The metric sums the differences in optimal path lengths between all nodes in the ground truth graph G and the proposal graph G’. For further details, see Blog1 and Blog2. For details on how to prepare data for the SpaceNet Road Detection and Routing Challenge, see Blog3.


1.0: Packages required (included in apls_environment.yml):

Osgeo (gdal, osr, ogr)



This script parses geojson labels into osmnx format for analysis. The valid_road_types field filters out only those road types desired (e.g. motorway, primary, etc.). This function requires the osgeo packages: gdal, osr, ogr.



Primary function for comparing ground truth and proposal graphs. The actual metric takes up a relatively small portion of this code (functions: single_path_metric, path_sim_metric, and compute_metric). Much of the code (functions: cut_linestring, get_closest_edge, insert_point, insert_control_points, great_graph_midpoints) is concerned with injecting nodes into graphs. We inject nodes (i.e. control points) into the graph at a predetermined distance along edges, and at the location nearest proposal nodes. Injecting nodes is essential to properly compare graphs, though it unfortunately requires quite a bit of code.

If is not installed (or more likely, if gdal is difficult to install), simply comment out the following functions in set_pix_coords and create_gt_graph.

The example in main() is self-contained, and demonstrates the process to insert midpoints into ground truth and proposal graphs, get shortest paths, and then compute the metric.

Instructions for downloading SpaceNet data can be found here.


3.0: Execution

Use conda to install all packages (currently tested with OSX and python 2)

cd /path/to/apls/src
conda env create -f apls_environment.yml   # to deactivate environment: source deactivate
source activate apls_environment
# for further details: python --help
#     primary option will 'test_method', experiment with: 
#		 python test_method=pkl
#		 python test_method=test_geojson
#		 python test_method=osmnx


4.0: Outputs

Running yields a number of plots in the example_output_ims directory, along with the APLS score

Alt text

Alt text



Run this script to create training masks with spacenet geojsons

cd /path/to/apls/src
# results are saved in example_output_ims/AOI_2_Vegas_Train