Evaluation of compare_geometries

[planemad]

Evaluated the compare geometries compare function w/ @bkowshik for various geometry transformation scenarios to see how the output scores compare relative to each issue.

New road, no geometry change
{"result:compare_geometries":{"cfVersion":2,"areaDelta":0,"centroidDisplacement":0,"geometryTransformation":3}} 

New road, one node dragged to double the length
{"result:compare_geometries":{"cfVersion":2,"areaDelta":38433.83745320714,"centroidDisplacement":581.8036084369106,"geometryTransformation":67082835.94906078}} 

New road, one node dragged to 1000 times the length
{"result:compare_geometries":{"cfVersion":2,"areaDelta":32931423208.534332,"centroidDisplacement":1196165.8065632868,"geometryTransformation":118174327210540240}} 

Mature road, one node dragged to double the length
{"result:compare_geometries":{"cfVersion":2,"areaDelta":38433.83745320714,"centroidDisplacement":581.8036084369106,"geometryTransformation":626106468.8579007}} 

Mature road, split in half
{"result:compare_geometries":{"cfVersion":2,"areaDelta":-21589.85576373099,"centroidDisplacement":243.93158052782456,"geometryTransformation":-147460533.91481057}} 

Mature road, displaced by 100m perpendicular to axis
{"result:compare_geometries":{"cfVersion":2,"areaDelta":-0.07723668643666315,"centroidDisplacement":111.2298332293394,"geometryTransformation":-240.54866504303104}} 

Mature road, displaced by 100m along axis
{"result:compare_geometries":{"cfVersion":2,"areaDelta":0,"centroidDisplacement":109.77333342690834,"geometryTransformation":3073.6533359534333}} 

Mature road, new node added and dragged by 1km
{"result:compare_geometries":{"cfVersion":2,"areaDelta":295886.4365296521,"centroidDisplacement":126.03898566162799,"geometryTransformation":1044210337.2744685}} 

Observations

• The area delta multiplier is strongly affecting the scores, we might need to use the square root to normalize its effect
• A node in the way getting displaced is weighted much much higher than the whole way getting displaced, even though both are definitely invalid transformations. We are currently not catching way displacements because of this https://github.com/mapbox/vandalism-dynamosm/issues/35
• We need to be able to differentiate a incorrect displacement or distortion from a legitimate improvement to a feature. This could possibly be done by comparing the number of nodes in the feature for changes
• When we are certain a transformation is invalid, it might be better to set an explicit flag rather than trying to quantify the problem as a numerical score.

Going to create some tickets to discuss possible improvements based on these findings.

cc @batpad @mikelmaron