Failed to retrieve a prediction response within the specified latency SLO

[lianzhang132]

Respected development team
I designed a simple example, here's the code, but why don't you get results when you run it?We look forward to and thank you for your reply。

#%%

def quick_sort(list):
if list == []:
return []
else:
first = list[0]
left = quick_sort([l for l in list[1:]if l < first])
right = quick_sort([l for l in list[1:] if l > first])
return left +[first] + right

#%%

from clipper_admin import ClipperConnection, DockerContainerManager
from clipper_admin.deployers import python as python_deployer

#%%

clipper_conn = ClipperConnection(DockerContainerManager())

#%%

clipper_conn.start_clipper()

Note that if the cluster is already created by start_clipper(), you can just connect to it using clipper_conn.connect() function.

#%%

python_deployer.create_endpoint(clipper_conn, "sort-quick", "integers", quick_sort,'-1',1,30000000)

#%%

def predict(addr, x, batch=False):
url = "http://%s/sort-quick/predict" % addr

if batch:
    req_json = json.dumps({'input_batch': x})
else:
    req_json = json.dumps({'input': list(x)})

headers = {'Content-type': 'application/json'}
start = datetime.now()
r = requests.post(url, headers=headers, data=req_json)
end = datetime.now()
latency = (end - start).total_seconds() * 1000.0
print("'%s', %f ms" % (r.text, latency))

#%%

import json ,requests
from datetime import datetime
addr = "127.0.0.1:1337"

x = '{"input": [1.1, 2.2, 3.3]}'

x = [9,2,6,4]
predict(addr,x,batch=False)

#%%

clipper_conn.stop_all()

#%%

[lianzhang132]

I also tried to increase the value of the SLO, but it didn't work

[lianzhang132]

@zoux86 看您提出过类似的issues