Add serialized warnings array to JSON output (#3588)

* [src/worker.cc] add warnings for deprecated auto-shorter

* [tyr/serializers.cc] function to serialize warnings

* [serializers.h] declare function for serializeWarnings

* [serializers.cc] refactor function for serializeWarnings

* [route_serializer_valhalla.cc] add warnings serializing function to route endpoint json response

* [serializer.cc] add warning type

* [warning] add warning for deprecated auto shorter

* [warning] add warning for deprecated hov costing

* [warning] add waring for deprecated auto_data_fix

* [warning] add warning for deprecated best_paths

* [tyr/serializers.cc] refactor serializeWarnings function

* [src/worker.cc] refactor warnings object into a function

* [tyr/serializer] convert warnings array object to flat array

* make warning_text const

* overload serializeWarnings function for baldr/json.h

* make warning_text a reference

* add function overload for serializeWarnings for other endpoints

* [isochrone_serializer.cc] add warnings to json output

* [locate_serializer.cc] add warning to locate endpoint json output

* [matrix_serializer.cc] add warnings to json response

* [height_serializer.cc] add warnings to json response

* [transit_available_serializer.cc] add warnings to json output

* [src/tyr/serializers.cc] refactor serializeWarnings function to remove double array

* [trace_serializer.cc] add warnings to json output

* refactor function to serialize warnings

* [src/worker.cc] make best_paths warning only valid for request with best_paths option

* prevent json response from containing an empty warnings array

* [test] add test for route endpoint warnings

* add array of deprecated costing options for test cases

* add test case for locate endpoint

* loop through deprecated costing options for route test case

* add test case for isochrone endpoint

* add test case for transit_available endpoint

* add test case for heights endpoint

* add test case for map_matching endpoints

* add test case for matrix endpoint

* change add warnings function to lambda function

* [routes] update documentation to include info about warnings

* [transit_available] update documentation with information about warnings

* [matrix] update documentation with info about warnings

* [map-matching] update documentation add information about warnings

* [locate] update documentation with information about warnings

* [isochrone] update documentation with information about warnings

* [elevation] update documentation with information about warnings

* code clean up - remove superflfous proto includes

* move lambda add_warnings function to worker.h

* update CHANGELOG.md with added enhancements

* remove loop indexes from route warnings array

* convert warnings function to a normal function

* add unordered map for warning pairs

* modify add_warnings function to utilizie unordered_map for warning pairs

* modify warnings messages to use unorderd_map for warning pairs

* add codes to serialized warnings

* add more context to the warnings array output.

* add more context to warnings array output.

* update to add more context to warnings array output

* add more context to warnings array output

* add more context to warnings array

* add more context to warnings json

* move function definition and warning codes to worker.cc

* rewrite tests to all use one map

* change variable name from i to costing for looping costing methods

* refactor test_warnings.cc to parametized gtest

* revert test cases

* remove warnings from transit endpoint

* remove warnings test for transit endpoint

* finish off PR

* lint & format; change some docs stuff

* untidy

* changelog update

* Update src/tyr/serializers.cc

Co-authored-by: Kevin Kreiser <kevinkreiser@gmail.com>

* Apply suggestions from code review

Co-authored-by: Kevin Kreiser <kevinkreiser@gmail.com>

* github UI conflict resolve failed format.sh

* raw strings instead of escaping quotes

* changelog

* typo

* revert gurka changes

* changelog

* wtf

* still..

* remove redundant include

Co-authored-by: nilsnolde <nilsnolde@proton.me>
Co-authored-by: Kevin Kreiser <kevinkreiser@gmail.com>
Co-authored-by: nilsnolde <nils.nolde@gmail.com>

CHANGELOG.md

@@ -122,6 +122,7 @@
    * ADDED: Add helpers for DirectedEdgeExt and save them to file in GraphTileBuilder [#3562](https://github.com/valhalla/valhalla/pull/3562)
    * ADDED: Fixed Speed costing option [#3576](https://github.com/valhalla/valhalla/pull/3576)
    * ADDED: Matrix action for gurka [#3793](https://github.com/valhalla/valhalla/pull/3793)
+   * ADDED: Add warnings array to response. [#3588](https://github.com/valhalla/valhalla/pull/3588)
 
 ## Release Date: 2021-10-07 Valhalla 3.1.4
 * **Removed**

docs/api/elevation/api-reference.md

@@ -107,6 +107,8 @@ The profile results are returned with the form of shape (shape points or encoded
 | `x coordinate` | The range or distance along the input locations. It is the cumulative distance along the previous latitiude, longitude coordinates up to the current coordinate. The x-value for the first coordinate in the shape will always be 0. |
 | `y coordinate` | The height or elevation of the associated latitude, longitude pair. The height is returned as `null` if no height data exists for a given location. |
 | `height` | An array of height for the associated latitude, longitude coordinates. |
+| `warnings` (optional) | This array may contain warning objects informing about deprecated request parameters, clamped values etc. | 
+
 
 ## Data sources
 

docs/api/isochrone/api-reference.md

@@ -53,7 +53,7 @@ The isochrone service uses the `auto`, `bicycle`, `pedestrian`, and `multimodal`
 
 In the service response, the isochrone contours are returned as [GeoJSON](http://geojson.org/), which can be integrated into mapping applications.
 
-The contours are calculated using rasters and are returned as either polygon or line features, depending on your input setting for the `polygons` parameter. If an isochrone request has been named using the optional `&id=` input, then the `id` is returned as a name property for the feature collection within the GeoJSON response. A `metric` attribute lets you know whether it's a `distance` or `time` contour.
+The contours are calculated using rasters and are returned as either polygon or line features, depending on your input setting for the `polygons` parameter. If an isochrone request has been named using the optional `&id=` input, then the `id` is returned as a name property for the feature collection within the GeoJSON response. A `metric` attribute lets you know whether it's a `distance` or `time` contour. A warnings array may also be included. This array may contain warning objects informing about deprecated request parameters, clamped values etc. | 
 
 See the [HTTP return codes](../turn-by-turn/api-reference.md#http-status-codes-and-conditions) for more on messages you might receive from the service.
 

docs/api/locate/api-reference.md

@@ -31,7 +31,7 @@ Because the locate service is designed to work in tandem with the route service
 
 If a request has been named using the optional `id` key, then this `id` key and value will be echoed in the JSON response object.
 
-The locate results are returned as a JSON array, with one JSON object per input location in the order specified. In `verbose` mode details about the streets and intersections includding mode of travel access, names, way ids, shape, side of street as well as the closest point to the input along these features will be returned. If `verbose` was not enabled only the closest point, way id and side of street will be returned.
+The locate results are returned as a JSON array, with one JSON object per input location in the order specified. In `verbose` mode details about the streets and intersections includding mode of travel access, names, way ids, shape, side of street as well as the closest point to the input along these features will be returned. If `verbose` was not enabled only the closest point, way id and side of street will be returned. A warnings array may also be included. This array may contain warning objects informing about deprecated request parameters, clamped values etc. | 
 
 Here are some sample results with `verbose` set to `false`:
 
@@ -61,6 +61,9 @@ Here are some sample results with `verbose` set to `false`:
         "percent_along": 1,
         "correlated_lon": -76.494987
       }
+    ],
+    "warnings": [
+      "hov costing is deprecated and will be turned into auto costing with hov2=true costing option"
     ]
   }
 ]
@@ -322,6 +325,9 @@ Here are some sample results with `verbose` set to `true`:
         "side_of_street": "neither",
         "correlated_lat": 40.313206
       }
+    ],
+    "warnings": [
+      "hov costing is deprecated and will be turned into auto costing with hov2=true costing option"
     ]
   }
 ]

docs/api/map-matching/api-reference.md

@@ -166,6 +166,7 @@ The `trace_attributes` results contains a list of edges and, optionally, the fol
 | `shape` | The [encoded polyline](../../decoding.md) of the matched path. |
 | `matched_points` | List of match results when using the `map_snap` shape match algorithm. There is a one-to-one correspondence with the input set of latitude, longitude coordinates and this list of match results. See the list of [matched point items](#matched-point-items) for details. |
 | `units` | The specified units with the request, in either kilometers or miles. |
+| `warnings`  | A warnings array. This array may contain descriptive text about notices of deprecated request parameters, clamped values etc. | 
 
 #### Edge items
 

docs/api/matrix/api-reference.md

@@ -72,6 +72,7 @@ These are the results of a request to the Time-Distance Matrix service.
 | `from_index` | The origin index into the locations array. |
 | `locations` | The specified array of lat/lngs from the input request.
 | `units` | Distance units for output. Allowable unit types are mi (miles) and km (kilometers). If no unit type is specified, the units default to kilometers. |
+| `warnings` (optional) | This array may contain warning objects informing about deprecated request parameters, clamped values etc. | 
 
 See the [HTTP return codes](/docs/api/turn-by-turn/api-reference.md#http-status-codes-and-conditions) for more on messages you might receive from the service.
 

docs/api/optimized/api-reference.md

@@ -58,6 +58,7 @@ These are the results of a request to the Optimized Route service.
 | `optimized_route` | Returns an optimized route path from point 'a' to point 'n'.  Given a list of locations, an optimized route with stops at each intermediate location exactly one time, always starting at the first location in the list and ending at the last location.|
 | `locations` | The specified array of lat/lngs from the input request.  The first and last locations in the array will remain the same as the input request.  The intermediate locations may be returned reordered in the response.  Due to the reordering of the intermediate locations, an `original_index` is also part of the `locations` object within the response.  This is an identifier of the location index that will allow a user to easily correlate input locations with output locations. |
 | `units` | Distance units for output. Allowable unit types are mi (miles) and km (kilometers). If no unit type is specified, the units default to kilometers. |
+| `warnings` (optional) | This array may contain warning objects informing about deprecated request parameters, clamped values etc. | 
 
 ## Error checking
 

docs/api/status/api-reference.md

@@ -17,3 +17,4 @@ If `"verbose": true` is passed as a parameter, the service will output the follo
 | `has_timezones`    | bool    | Whether the current tileset was built using the timezone database. |
 | `has_live_traffic` | bool    | Whether live traffic tiles are currently available. |
 | `bbox`             | object  | GeoJSON of the tileset extent. |
+| `warnings` (optional) | array | This array may contain warning objects informing about deprecated request parameters, clamped values etc. | 

docs/api/transit-available/api-reference.md

@@ -40,5 +40,6 @@ These are the results of a request to the Transit Availability service.
 | `transit_available` | Returns a boolean value for if transit is available, along with the input a list of locations and radius used for the check.|
 | `istransit` | A boolean value for whether or not transit exists for a particular location and radius.
 | `locations` | The specified array of lat/lngs from the input request.  Locations may also contain an optional radius. |
+| `warnings`  | A warnings array. This array may contain descriptive text about notices of deprecated request parameters, clamped values etc. |  
 
 See the [HTTP return codes](/turn-by-turn/api-reference.md#http-status-codes-and-conditions) for more on messages you might receive from the service.

docs/api/turn-by-turn/api-reference.md

@@ -330,6 +330,7 @@ Basic trip information includes:
 | `units` | The specified units of length are returned, either kilometers or miles. |
 | `language` | The language of the narration instructions. If the user specified a language in the directions options and the specified language was supported - this returned value will be equal to the specified value. Otherwise, this value will be the default (en-US) language. |
 | `locations` | Location information is returned in the same form as it is entered with additional fields to indicate the side of the street. |
+| `warnings` (optional) | This array may contain warning objects informing about deprecated request parameters, clamped values etc. | 
 
 The summary JSON object includes:
 
@@ -342,6 +343,7 @@ The summary JSON object includes:
 | `max_lat` | Maximum latitude of a bounding box containing the route. |
 | `max_lon` | Maximum longitude of a bounding box containing the route. |
 
+
 ### Trip legs and maneuvers
 
 A `trip` contains one or more `legs`. For *n* number of `break` locations, there are *n-1* legs. `Through` locations do not create separate legs.

docs/api/turn-by-turn/overview.md

@@ -6,7 +6,7 @@ The Valhalla route service (a.k.a. turn-by-turn) is an open-source routing servi
 
 When you [request a route](api-reference.md#inputs-of-a-route), you are sending and receiving [JSON](https://en.wikipedia.org/wiki/JSON), which is a human-readable text format. In the JSON array, you need to specify the [locations](api-reference.md#locations) to visit on the route, the [costing model](api-reference.md#costing-models) that represents the mode of travel, such as car or bicycle, and your API key. The location coordinates, given in decimal degrees, can come from many input sources, such as a GPS location, a point or a click on a map, a geocoding service such as [Mapbox Geocoding](https://docs.mapbox.com/api/search/#geocoding), and so on. Costing methods can have several options that can be adjusted to develop the the route path and estimate the time along the path.
 
-The service [route results](api-reference.md#outputs-of-a-route) provide details about the trip, including locations, a summary with basic information about the entire trip, and a list of legs. Each leg has its own summary, a shape, which is an encoded polyline of the route path, and a list of maneuvers. These maneuvers provide written narrative instructions, plus verbal alerts that can be used as audio guidance in navigation apps.
+The service [route results](api-reference.md#outputs-of-a-route) provide details about the trip, including locations, a summary with basic information about the entire trip and a list of legs. Each leg has its own summary, a shape, which is an encoded polyline of the route path, and a list of maneuvers. These maneuvers provide written narrative instructions, plus verbal alerts that can be used as audio guidance in navigation apps.
 
 The JSON returned from the route query can be drawn on a map and shown as instructions for maneuvers along the route. You can [display Valhalla routes](add-routing-to-a-map.md) on web and mobile maps.
 

src/tyr/height_serializer.cc

@@ -94,6 +94,11 @@ std::string serializeHeight(const Api& request,
     json->emplace("id", request.options().id());
   }
 
+  // add warnings to json response
+  if (request.info().warnings_size() >= 1) {
+    json->emplace("warnings", serializeWarnings(request));
+  }
+
   std::stringstream ss;
   ss << *json;
   return ss.str();

src/tyr/isochrone_serializer.cc

@@ -132,6 +132,11 @@ std::string serializeIsochrones(const Api& request,
     feature_collection->emplace("id", request.options().id());
   }
 
+  // add warnings to json response
+  if (request.info().warnings_size() >= 1) {
+    feature_collection->emplace("warnings", serializeWarnings(request));
+  }
+
   std::stringstream ss;
   ss << *feature_collection;
 

src/tyr/matrix_serializer.cc

@@ -132,6 +132,12 @@ json::MapPtr serialize(const Api& request,
   if (options.has_id_case()) {
     json->emplace("id", options.id());
   }
+
+  // add warnings to json response
+  if (request.info().warnings_size() >= 1) {
+    json->emplace("warnings", valhalla::tyr::serializeWarnings(request));
+  }
+
   return json;
 }
 } // namespace valhalla_serializers

src/tyr/route_serializer_osrm.cc

@@ -1726,6 +1726,11 @@ std::string serialize(valhalla::Api& api) {
   json->emplace(options.action() == valhalla::Options::trace_route ? "matchings" : "routes",
                 std::move(routes));
 
+  // get serialized warnings
+  if (api.info().warnings_size() >= 1) {
+    json->emplace("warnings", serializeWarnings(api));
+  }
+
   std::stringstream ss;
   ss << *json;
   return ss.str();

src/tyr/route_serializer_valhalla.cc

@@ -538,6 +538,11 @@ std::string serialize(const Api& api) {
     // summary time/distance and other stats
     summary(api, i, writer);
 
+    // get serialized warnings
+    if (api.info().warnings_size() >= 1) {
+      valhalla::tyr::serializeWarnings(api, writer);
+    }
+
     writer.end_object(); // trip
 
     // leave space for alternates by closing this one outside the loop

src/tyr/serializers.cc

@@ -157,6 +157,25 @@ void openlr(const valhalla::Api& api, int route_index, rapidjson::writer_wrapper
   writer.end_array();
 }
 
+void serializeWarnings(const valhalla::Api& api, rapidjson::writer_wrapper_t& writer) {
+  writer.start_array("warnings");
+  for (const auto& warning : api.info().warnings()) {
+    writer.start_object();
+    writer("code", warning.code());
+    writer("text", warning.description());
+    writer.end_object();
+  }
+  writer.end_array();
+}
+
+json::ArrayPtr serializeWarnings(const valhalla::Api& api) {
+  auto warnings = json::array({});
+  for (const auto& warning : api.info().warnings()) {
+    warnings->emplace_back(json::map({{"code", warning.code()}, {"text", warning.description()}}));
+  }
+  return warnings;
+}
+
 std::string serializePbf(Api& request) {
   // if they dont want to select the parts just pick the obvious thing they would want based on action
   PbfFieldSelector selection = request.options().pbf_field_selector();

src/tyr/trace_serializer.cc

@@ -544,6 +544,12 @@ std::string serializeTraceAttributes(
     ++route;
   }
   writer.end_array();
+
+  // add warnings to json response
+  if (request.info().warnings_size() >= 1) {
+    valhalla::tyr::serializeWarnings(request, writer);
+  }
+
   writer.end_object();
   return writer.get_buffer();
 }

src/worker.cc

@@ -138,6 +138,26 @@ const std::unordered_map<unsigned, valhalla::valhalla_exception_t> error_codes{
     {503, {503, "Leg count mismatch", 400, HTTP_400, OSRM_INVALID_URL, "wrong_number_of_legs"}},
 };
 
+// unordered map for warning pairs
+const std::unordered_map<int, std::string> warning_codes = {
+  // 1xx is for deprecations
+  {100, R"(auto_shorter costing is deprecated, use "shortest" costing option instead)"},
+  {101,
+    R"(hov costing is deprecated, use "include_hov2" costing option instead)"},
+  {102, R"(auto_data_fix is deprecated, use the "ignore_*" costing options instead)"},
+  {103, R"(best_paths has been deprecated, use "alternates" instead)"}
+};
+
+// function to add warnings to proto info object
+void add_warning(valhalla::Api& api, int code) {
+  auto message = warning_codes.find(code);
+  if (message != warning_codes.end()) {
+    auto* warning = api.mutable_info()->mutable_warnings()->Add();
+    warning->set_description(message->second);
+    warning->set_code(message->first);
+  }
+}
+
 // clang-format on
 
 rapidjson::Document from_string(const std::string& json, const valhalla_exception_t& e) {
@@ -616,6 +636,9 @@ void from_json(rapidjson::Document& doc, Options::Action action, Api& api) {
   // auto_shorter is deprecated and will be turned into
   // shortest=true costing option. maybe remove in v4?
   if (costing_str == "auto_shorter") {
+
+    add_warning(api, 100);
+
     costing_str = "auto";
     rapidjson::SetValueByPointer(doc, "/costing", "auto");
     auto json_options = rapidjson::GetValueByPointer(doc, "/costing_options/auto_shorter");
@@ -628,6 +651,10 @@ void from_json(rapidjson::Document& doc, Options::Action action, Api& api) {
   // hov costing is deprecated and will be turned into auto costing with
   // include_hov2=true costing option.
   if (costing_str == "hov") {
+
+    // add warning for hov costing
+    add_warning(api, 101);
+
     costing_str = "auto";
     rapidjson::SetValueByPointer(doc, "/costing", "auto");
     auto json_options = rapidjson::GetValueByPointer(doc, "/costing_options/hov");
@@ -640,6 +667,10 @@ void from_json(rapidjson::Document& doc, Options::Action action, Api& api) {
   // auto_data_fix is deprecated and will be turned into
   // ignore all the things costing option. maybe remove in v4?
   if (costing_str == "auto_data_fix") {
+
+    // warning for auto data fix
+    add_warning(api, 102);
+
     costing_str = "auto";
     rapidjson::SetValueByPointer(doc, "/costing", "auto");
     auto json_options = rapidjson::GetValueByPointer(doc, "/costing_options/auto_data_fix");
@@ -1037,6 +1068,10 @@ void from_json(rapidjson::Document& doc, Options::Action action, Api& api) {
     }
   }
 
+  // add warning for deprecated best_paths
+  if (rapidjson::get_optional<uint32_t>(doc, "/best_paths")) {
+    add_warning(api, 103);
+  }
   // deprecated best_paths for map matching top k
   auto best_paths = std::max(uint32_t(1), rapidjson::get<uint32_t>(doc, "/best_paths", 1));
 

test/gurka/test_warnings.cc

@@ -0,0 +1,16 @@
+#include "gurka.h"
+#include <gtest/gtest.h>
+#include <vector>
+
+using namespace valhalla;
+
+TEST(Warning, DeprecatedParams) {
+  for (std::string costing : {"auto_shorter", "hov", "auto_data_fix"}) {
+    Api request;
+    std::string request_str =
+        R"({"best_paths": 2, "locations": [{"lat": 0.0, "lon": 0.0},{"lat": 1.0, "lon": 1.0}], "costing": ")" +
+        costing + R"("})";
+    ParseApi(request_str, Options::route, request);
+    EXPECT_EQ(request.info().warnings_size(), 2);
+  }
+}

valhalla/tyr/serializers.h

@@ -117,6 +117,13 @@ void openlr(const valhalla::Api& api, int route_index, rapidjson::writer_wrapper
  */
 std::string serializePbf(Api& request);
 
+/**
+ * @brief Turns warnings into json
+ * @param request The protobuf warnings object
+ * @return json string
+ */
+void serializeWarnings(const valhalla::Api& api, rapidjson::writer_wrapper_t& writer);
+baldr::json::ArrayPtr serializeWarnings(const valhalla::Api& api);
 } // namespace tyr
 } // namespace valhalla
 

valhalla/worker.h

@@ -74,6 +74,10 @@ void ParseApi(const prime_server::http_request_t& http_request, Api& api);
 #endif
 
 std::string serialize_error(const valhalla_exception_t& exception, Api& options);
+
+// function to add warnings to proto info object
+void add_warning(valhalla::Api& api, unsigned code);
+
 #ifdef HAVE_HTTP
 prime_server::worker_t::result_t serialize_error(const valhalla_exception_t& exception,
                                                  prime_server::http_request_info_t& request_info,