meili_request

[saraefaf1]

I tried to send a request with meili, the Valhalla server also runs virtually in the vmbox at the same time, but I do not receive a response. any help is really appreciated. Thanks.

import pandas as pd
import requests
import json

waypoints=pd.read_csv('waypoints.csv',names=["TripID","WaypointSequence","CaptureDate","Latitude","Longitude","SegmentID","ZoneName","FRC","DeviceId","RawSpeed","RawSpeedMetric"])
#ds=waypoints.groupby('TripID').count()
ds=waypoints.groupby('TripID')

df_sampel = ds.get_group('3dba9f6bd6cc725263cf5b4dbb84d9f3')
pd.to_datetime(df_sampel['CaptureDate'])

df_sampel=df_sampel[['Longitude','Latitude','CaptureDate']]
df_sampel.columns=['lon','lat','time']

df_sampel_sortet=df_sampel.sort_values('time',axis=0)

data = df_sampel_sortet.to_json(orient='records')

#meili_coordinates = df_trip.to_json(orient='records')
meili_coordinates =data

Providing needed data for the body of Meili's request

meili_head = '{"shape":'

Those are parameters that you can change according to the Meili's documentation

meili_tail = ""","search_radius": 150, "shape_match":"map_snap", "costing":"auto", "format":"osrm"}"""

Combining all the string into a single request

meili_request_body = meili_head + meili_coordinates + meili_tail

The URL of our local Valhalla server

"trace_route" means that we're sening a request to Meili

url = "http://0.0.0.0:8002/trace_route"

Providing headers to the request

headers = {'Content-type': 'application/json'}

We need to send our JSON as a string

meili_request_body=json.loads(meili_request_body)
data = json.dumps(meili_request_body)
try:
# Sending a request
#print(data)
#print(url)
#print(headers)
r = requests.post(url,data=data,headers=headers)
#print(r.text)
except requests.Timeout as e:
print("OOPS!! Timeout Error")
print(str(e))
renewIPadress()
#continue

except requests.ConnectionError as e:
print("OOPS!! Connection Error. Make sure you are connected to Internet. Technical Details given below.\n")
print(str(e))
renewIPadress()
#continue

except requests.RequestException as e:
print("OOPS!! General Error")
print(str(e))
renewIPadress()
#continue
except KeyboardInterrupt:
print("Someone closed the program")

Checking if the response from Meili was correct

if r.status_code == 200:
# Parsing the JSON response
response_text = json.loads(r.text)

# There are a lot more information that we got from Meili
# but I'm interested in just 'tracepoints'
resp = str(response_text['tracepoints'])

# This is a replacement to distinguish single None's in a row
# from "waypoint_index" being None
resp = resp.replace("'waypoint_index': None", "'waypoint_index': '#'")
resp = resp.replace("None", "{'matchings_index': '#', 'name': '', 'waypoint_index': '#', 'alternatives_count': 0, 'distance': 0, 'location': [0.0, 0.0]}")

# This is to make it a valid JSON
resp = resp.replace("'", '"')
resp = json.dumps(resp)
resp = json.loads(resp)

# Reading our JSON to a Pandas DataFrame
df_response = pd.read_json(resp)

# Saving the columns that I actually need
df_response = df_response[['name', 'distance', 'location']]
df_response
# Merging our initial data with the response from Meili by index
df_trip_optimized = pd.merge(waypoints, df_response, left_index=True, right_index=True)

Processing through our dataframe

for i, row in df_trip_optimized.iterrows():
# Parsing a 'location' data that we got from Meili
# to get 2 columns of optimized longitude and latitude
df_trip_optimized.at[i, 'lon_optimized'] = df_trip_optimized.at[i,'location'][0]
df_trip_optimized.at[i, 'lat_optimized'] = df_trip_optimized.at[i,'location'][1]
# When either longitude or latitude that we got from Meili equals to 0
# we assume that Meili couldn't find any roads with given parameters
# therefore it's safe to save our previous longitude and latitude
if df_trip_optimized.at[i, 'lon_optimized'] == 0.0:
df_trip_optimized.at[i, 'lon_optimized'] = df_trip_optimized.at[i,'lon']
df_trip_optimized.at[i, 'lat_optimized'] = df_trip_optimized.at[i,'lat']

Done processing, can just drop the location column

df_trip_optimized = df_trip_optimized.drop(['location'], 1)

[kevinkreiser]

There is a ton of code here but from the beginning of this issue it sounded more like the code isnt the problem but rather that you dont get a response from the map matching service?

If this is the case please tell us how you are running the service. It sounded like you were saying its running in a VM, is the VM a docker container? Please elaborate on how the service is running, can you hit the service in a web browser just to be sure its answering requests, after that we can move on to what you want to do in code.

[saraefaf1]

I'm almost sure the server is working because I tested it before, I run the Valhalla server in ubuntu VMbox.

[kevinkreiser]

Valhalla server also runs virtually in the vmbox at the same time, but I do not receive a response

if you know for sure its working then the above statement doesnt make sense to me. here's a python example of hitting the server:

https://github.com/OpenStreetMapSpeeds/conflation/blob/99042927a0675db9e1d570a1f31750c68a633498/conflation/map_matching/valhalla.py#L139-L147

here base_url is something like http://localhost:8002/trace_attributes. if your server is actually properly running and you can connect to it then you will get a response even if your request was completely messed up (in terms of formatting) you'd still get a response. it wouldnt be a 200, it would probably be a 400 though if the problem is with your request.

[saraefaf1]

now,i have a new error: "code":"NoSegment","message":"One of the supplied input coordinates could not snap to street segment."
or 400::no suitable edges near location request.
how can modify an Edge for my Request?

[kevinkreiser]

this means that we cant find any roads near your input location. this could either be:

1. you dont have your data properly loaded so that no input locations are near anything because nothign is loaded
2. or your input is erroneous, did you switch lat and lon for example?

[saraefaf1]

i have already snapped all thae points with help folium and can see all of them of the map

[saraefaf1]

id,lat,lon,time
6295905 50.837600 7.094700 2021-03-01T08:20:12.000Z
3403424 50.841238 7.095817 2021-03-01T08:20:27.000Z
4080409 50.842390 7.096191 2021-03-01T08:20:32.000Z
2884438 50.845809 7.097280 2021-03-01T08:20:47.000Z
6623424 50.851806 7.097788 2021-03-01T08:21:12.000Z
4887633 50.856086 7.097290 2021-03-01T08:21:37.000Z
5365309 50.857532 7.096971 2021-03-01T08:21:47.000Z
1764050 50.858139 7.096886 2021-03-01T08:21:52.000Z
3779209 50.858644 7.096808 2021-03-01T08:21:57.000Z
3405408 50.859432 7.096692 2021-03-01T08:22:02.000Z

[kevinkreiser]

show me your valhalla formatted request, if that looks good then it must be that you dont have the data loaded

[saraefaf1]

data = d3.to_json(orient='records')
meili_tail = ""","search_radius": 130, "shape_match":"map_snap", "costing":"auto", "format":"osrm"}"""
meili_request_body =data
url = "http://192.168.1.127:8002/trace_route"
headers = {'Content-type': 'application/json'}
data ='{"shape":'+ meili_request_body+meili_tail
d1=json.loads(data)
d2=json.dumps(d1)
r = requests.post(url,data=d2,headers=headers)

[kevinkreiser]

yeah can you please add

print(json.dumps(data))

and paste the result here

[saraefaf1]

"{"shape":[{"lat":50.8376,"lon":7.0947,"time":"2021-03-01T08:20:12.000Z"},{"lat":50.841238,"lon":7.095817,"time":"2021-03-01T08:20:27.000Z"},{"lat":50.84239,"lon":7.096191,"time":"2021-03-01T08:20:32.000Z"},{"lat":50.845809,"lon":7.09728,"time":"2021-03-01T08:20:47.000Z"},{"lat":50.851806,"lon":7.097788,"time":"2021-03-01T08:21:12.000Z"},{"lat":50.856086,"lon":7.09729,"time":"2021-03-01T08:21:37.000Z"},{"lat":50.857532,"lon":7.096971,"time":"2021-03-01T08:21:47.000Z"},{"lat":50.858139,"lon":7.096886,"time":"2021-03-01T08:21:52.000Z"},{"lat":50.858644,"lon":7.096808,"time":"2021-03-01T08:21:57.000Z"},{"lat":50.859432,"lon":7.096692,"time":"2021-03-01T08:22:02.000Z"}],"search_radius": 130, "shape_match":"map_snap", "costing":"auto", "format":"osrm"}"
{"shape": [{"lat": 50.8376, "lon": 7.0947, "time": "2021-03-01T08:20:12.000Z"}, {"lat": 50.841238, "lon": 7.095817, "time": "2021-03-01T08:20:27.000Z"}, {"lat": 50.84239, "lon": 7.096191, "time": "2021-03-01T08:20:32.000Z"}, {"lat": 50.845809, "lon": 7.09728, "time": "2021-03-01T08:20:47.000Z"}, {"lat": 50.851806, "lon": 7.097788, "time": "2021-03-01T08:21:12.000Z"}, {"lat": 50.856086, "lon": 7.09729, "time": "2021-03-01T08:21:37.000Z"}, {"lat": 50.857532, "lon": 7.096971, "time": "2021-03-01T08:21:47.000Z"}, {"lat": 50.858139, "lon": 7.096886, "time": "2021-03-01T08:21:52.000Z"}, {"lat": 50.858644, "lon": 7.096808, "time": "2021-03-01T08:21:57.000Z"}, {"lat": 50.859432, "lon": 7.096692, "time": "2021-03-01T08:22:02.000Z"}], "search_radius": 130, "shape_match": "map_snap", "costing": "auto", "format": "osrm"}

[kevinkreiser]

I have no idea what will happen with those timestamps. The api takes epoch times for per location timestamps

[saraefaf1]

if times does not necessary,for map matching,can it remove from df?

[kevinkreiser]

thats right, its required you can remove it. it helps to make the match better but it is optional

[saraefaf1]

i have yet same problem:

"{"shape":[{"lat":50.8376,"lon":7.0947},{"lat":50.841238,"lon":7.095817},{"lat":50.84239,"lon":7.096191},{"lat":50.845809,"lon":7.09728},{"lat":50.851806,"lon":7.097788},{"lat":50.856086,"lon":7.09729},{"lat":50.857532,"lon":7.096971},{"lat":50.858139,"lon":7.096886},{"lat":50.858644,"lon":7.096808},{"lat":50.859432,"lon":7.096692}],"search_radius": 130, "shape_match":"map_snap", "costing":"auto", "format":"osrm"}"
{"code":"NoSegment","message":"One of the supplied input coordinates could not snap to street segment."}

[kevinkreiser]

ok great can you confirm that you have actually loaded tile data in the service? your config file should have either tile_dir or tile_extract set to a path that contains a directory of tiles or a tar file extract respectively.

[saraefaf1]

i have 2 folder in path Scripts folder, one ist conf folder in wich i have only valhalla.json and another folder name is valhalla-tiles.
in conf folder ,there is not tile_dir or tile_extract.how can i make it?

[nilsnolde]

before going any further: can you pls make this fully reproducible, so we're not poking around in the dark:

1. show us where are which files, so in your root folder of the project do a tree . and paste the output (not the valhalla code, we're only interested where is the valhalla.json and where are the valhalla tile files located)
2. show us the exact commands you used to a) create the valhalla tiles and b) running the valhalla service
3. paste the contents of the valhalla.json mjolnir member (not the full thing pls)

Also make sure you format your answer properly pls, esp for code use 3 backticks: https://guides.github.com/features/mastering-markdown/

[saraefaf1]

thank you for your Help, the problem was my commands to create the valhalla tiles and now i have a correct response.