Weighting support via yaml configuration

[karussell]

fixes #1776
fixes #223

Note

Disclaimer: custom profiles were an alpha feature in version 1.0 and 2.0.

You can try this feature in the linked small tutorial. Please note that the format changed in version 3.0 - see here for the pull request and the blog post with examples which is always updated.

Usage

First you need to define a custom profile:

profiles:
  - name: car
    vehicle: car
    weighting: custom

Then you can use this profile in a custom request. You query a json towards the new endpoint /route-custom. In the following section we use yaml to allow comments (you can convert it with this node js tool or this online tool):

# query parameters:
points: [[10.373926, 52.042989], [10.384043, 52.042289]]
details: [road_class, street_name]

# pick the previously defined custom profile
profile: car

# The formula is defined in CustomWeighting, where the speed_factor can be used to decrease speed.
# The minimum value is 0 and the maximum is 1. If two conditions are satisfied the values are
# multiplied e.g. if road_class==motorway and road_environment==tunnel, then the resulting speed is
# average_speed*0.85*0.9.
speed_factor:
  road_environment:
    tunnel: 0.85
  # trucks should be a bit slower on certain road classes compared to the 'base' car
  # a more compact JSON-way to list the properties, fully YAML-compatible:
  road_class: { motorway: 0.85, primary: 0.9 }

# You can lower the average speed for certain conditions via this max_speed entry. The speed_factor is applied before
# this operation. See the following example that sets the maximum speed to 95km/h for motorways.
max_speed:
  road_class:
    motorway: 95
    residential: 30
  road_environment:
    bridge: 85

# Trucks are slower so limit all speed to this value. In km/h
# If none of the conditions in the map above apply use the fallback, if specified:
max_speed_fallback: 100

# This term changes the influence of the distance. I.e. longer roads get a higher cost.
# The distance_influence is independent of the edge properties and does not influence the ETA. The default is 70 (seconds/1km).
# Let's assume a route that takes 1000sec and is 10km long, then a value of 30 means that I would like to drive maximum 11km
# to reduce the travel time to 970sec or 12km to reduce it to 940sec.
distance_influence: 90

# Now we want to set a preference without changing the taken time. The default priority is 1 and does not change something.
# Higher than 1 is not possible directly, only via the catch-all key "*", see road_class where motorways are preferred. Lower than 1 will avoid it.
# If two conditions are met the values will be multiplied. The minimum value is 0 (block access) the maximum value is 100.
priority:
  road_class:
    motorway: 1.0
    residential: 0.4
    "*": 0.9
    # and if you do not want that tracks go over tracks just exclude them via
    # track: 0
  # let's assume we transport gas: so NEVER go on restricted roads with hazmat==no
  hazmat: { no: 0 }
  # avoid destination-only roads
  road_access: { destination: 0.1 }
  # avoid turns if possible and links are one simple indication for that
  road_class_link:
    true: 0.5
  # avoid toll roads
  toll: { no: 1, "*": 0.5 }
  # avoid a certain area
  area_custom1: 0.5
  max_weight: { "<4.5": 0 }
  max_height: { "<3.8": 0 }
  max_width: { "<2.5": 0 }

areas:
  custom1:
    type: "Feature"
    geometry: { type: "Polygon", coordinates: [[[13.722, 51.053], [13.722, 51.055], [13.731, 51.055], [13.731, 51.053], [13.722, 51.053]]] }

Or a cargo bike:

profile: bike

# let's assume cargo bikes are e-bikes and we should lower speed to prefer shorter and not faster routes automatically
max_speed:
  road_class:
    primary: 28

# if no other condition matches
max_speed_fallback: 25

priority:
  # exclude steps
  road_class:
    steps: 0
  surface:
    sand: 0.5
  # prefer better tracks
  track_type: { other: 1.0, grade1: 1.0, "*": 0.9 }
  # prefer official bike routes
  bike_network: { other: 0.5 }
  # avoid all situations where we have to get off the bike
  get_off_bike:
    "true": 0.5
  max_height:
    "<2.3": 0
  max_width:
    "<1.2": 0

Now the interesting part is that you can try this out in the demo UI and when you are happy you can use the exact same content to create a file (e.g. truck.yml) and use it for the LM or CH preparation to make the query much faster, like so:

profiles:
 - name: truck
   vehicle: car
   weighting: custom
   custom_model_file: path/to/truck.yml
 - name: cargo_bike
   vehicle: bike
   weighting: custom
   custom_model_file: path/to/cargo_bike.yml

To query such a prepared custom profile you specify e.g. profile=truck. If you want to customize your request further you use A* or LM and the /route-custom endpoint. If no further customization is required or you want to use a CH-prepared custom profile you use the /route endpoint.

When you try this out in the UI you currently have to include custom weighting and potentially disabling turn cost: http://localhost:8989/maps/?point=51.060063%2C13.714714&point=51.064378%2C13.756599&weighting=custom&turn_costs=false

• basic changes of the Weighting already work
• examples and description is available in truck.yml, RouteResourceTest and CustomWeightingRouteResourceTest. Other examples could be: wheelchair (base:foot), electric_bike, cargo_bike (base:electric_bike -> which is not a FlagEncoder!), electric_car (base:car), bike that prefers official bike routes much more, fire_truck (base:truck), scooter (speed limited car?)
• query via json or yaml
• no need for a jackson dependency in core
• implement DelayFlexConfig see below
• for now remove delay as there are similar problems like for elevation (needs to be splitted for virtual edges) otherwise it sums up if there are several query points on a single edge.
• -d for curl does not work for yaml as it strips line breaks. Instead use --data-binary
• make FlexModelWeighting possible for CH and LM preparation too, without introducing a jackson dependency to core (due to Android)
• priority is implemented wrong => it should be priority=1/factor
• support for weight, height, width limitations
• more "FlexModelWeighting" features like shown here should follow in separate PRs
• merge FlexResource into RouteResource but only support it for POST
• simple UI support like here
• throw illegal arg exception if we use max_height but this is not supported from the storage.
• documentation in config-example.yml and docs/ maybe
• rename weight -> max_vehicle_weight etc
• Yaml is much more standard & "better", but even our simplistic gscript implementation had a much better error reporting. Maybe also allow ScriptedWeighting via some interfaces and bring it other languages. For now yaml is ok. Later we might add a ScriptedWeighting.
• if speed value is too slow => rounding to zero => is blocking desired? should not happen as we do not store the weight again into an EncodedValue
• different/separate formula for calcWeight vs. calcTime? probably not now
• add difference of average_speed handling (first wins) vs. speed_factor handling (all multiplied) to docs
• throw error if value in average_speed is higher than max_speed
• add custom request to Measurement
• extend from AbstractWeighting
• A very significant route change and query speed difference between normal "car" requests and a custom weighting with base: car is the default of distance_factor being 1 instead of 0. I.e. a normal car request takes 1100ms and the same as custom request with just "base: car" takes only 350ms. For now changed this default to 0.07 (copied from ShortFastestWeighting)
• replace average_speed with a "max speed" variable or even remove it completely? The average_speed is not that useful. The speed value could be from traffic sources and we would completely overwrite it by a rather simple condition like road_class==motorway. Also it should be probably more a max_average_speed (instead of max speed)
• before merging we need Improve edge filtering for LocationIndex #729
• CustomerWeighting should always increase weight (normalize priority and factor)
• CustomWeighting does not work with LM (not in Measurement and not in RouteResource)
• See issue Select CH profiles by name ? #1708. Instead of base should we always pick the vehicle from the request? (this won't work for CH.) -> Do not use vehicle as it will collide with GHRequest.vehicle.
• avoid nesting the model in the request because using the profile in a request or a request for a profile is ugly and only possible via editing. But then a separate endpoint is required?
• throw exception if vehicle and weighting is specified in URL but does not match -> not needed if good documentation
• introduce distance_factor map
• web UI: keep flex config between requests (update: this only happens when setting weighting=flex as url parameter)
• "sum" of terms instead of "multiplication" would make more sense, but then "preferring" a certain attribute is no longer as easy and often a requirement if we want to keep the yaml expressions simple (i.e. no complex boolean algebra)
• check again about A* correctness (min weight -> priority and speed range)
• is distance_factor map really necessary or too complicated too understand the nuances to "inverse priority"? -> for now remove it
• allow boolean encoded values too like road_class_link or get_off_bike. Would be great to use this for "avoiding".
• support block_area in the CustomModel
• simple turn cost support for query -> already works via root-level fields populated GHRequest.hints (e.g. disabling works via edge_based: false)
• switch to snake case is a bit ugly as current config.ymls stop working
• Avoid error: "You can try disabling LM by setting lm.disable=true"
• try using country enum
• remove snake case as it is already in PR no need to explicitely list the snake case methods #1918
• for fun try "avoid left turns" via per-request geometry evaluation 38b62f0
• remove newly introduced name parameter from AbstractWeighting
• change formula to either (const_distance_term + 1/v/prio)*d (no multiplication of prio with const_distance_term) or (const_distance_term + k/v + sum(some_map))*d (instead of priority multiplication use summation)
• replace mixins with SNAKE-enabled jackson parser
• if a CustomModel abc is known to GraphHopper and then there is a CustomRequest with profile=abc but a few parameters have changed, then it should be possible to merge them. Maybe in CustomWeighting.prepareRequest?
• is it ugly that we do profile=custommodel against /route but if it is a CustomRequest it is against /route-custom. Especially if it is a CustomRequest with profile=custommodel (see point above) it gets strange.
• turn cost support for CH/LM preparation -> should be already done via profiles
• CustomModel.merge should multiply (or pick maximum) to keep LM-correctness
• solve Select CH profiles by name ? #1708
• change endpoint to /route-custom
• add JSON support for file import and remove yaml endpoint
• formula change Change custom weighting formula #2021
• merge different ranges of DecimalEncodedValue
• update docs
• revert FlagEncoder changes

Later:

• solve Move access flag checks into Weighting #1835
• CustomRequest for Isochrone & SPT endpoint (POST request)
• preliminary & simple tests show at least 20% slower CH preparation even for a very simple FlexModel. -> minor improvements to 10% - but for the simple case only! Can we somehow cache calls to edge.get(roadClass) so that at least if only road_class is used for the different FlexConfigs it does not get slower?
• Cache EncodedValues in a later issue
• try to get car turn restrictions working for a bike model (car.turn_cost)
• web UI: show more useful error messages?
• allow DoubleEncodedValue as value on the "right side" e.g. to directly use an EncodedValue as speed_factor. This is a bit complicated as we do not know the maximum value and cannot ensure correctness for LM.
• allow != and &&
• later allow a special shorter notation ala
  speed_factor: { road_class: { motorway||trunk: 1.1 }} instead of
  speed_factor: { road_class: { motorway: 1.1, trunk: 1.1 }}?
• elevation. Let us assume we store the up and down elevation in EncodedValues then this is more involved for a QueryGraph as we need to split this for virtual edges like it is currently done also for the distance. Keep in mind that there are different use cases for elevation: one for weight-only (electric car) and one where weight and time is changed (bike that avoid hills). Also do delay (time offset) later as same problems like for elevation Introduce edge-length dependent EncodedValues #1898
• avoid changing a country
• turn cost support in Weighting class (see Remove TurnWeighting #1820) and also for importFlexModels
• ScriptedWeighting

[karussell]

I found a really nice use case with the more generic >number schema we introduced recently. We cannot only apply this to max_weight or other vehicle properties but e.g. also to car.average_speed, and this can be used to avoid slower speed areas like city (ok, just an ugly workaround for maybe some future property like is_in_city) but it can also be used to avoid bike routes on roads with high (car) speed which was currently only possible within the FlagEncoder.