AttributeError: 'Middleware' object has no attribute 'options'

[echzhai]

Search before asking

• I searched the issues and found no similar issues.

KubeRay Component

apiserver

What happened + What you expected to happen

ray serve model deployed successful. the status is runing.
but can not access port 8000 to access the deployed model.
and there is error:
Exception raised in creation task: The actor died because of an error raised in its creation task, �[36mray::SERVE_CONTROLLER_ACTOR:SERVE_PROXY_ACTOR-ef52d5aa58d16967c8c193509c9bd2658fe00e137c795e7f30b7142e:ProxyActor.init()�[39m (pid=1179, ip=192.168.106.226, actor_id=e9b35e7b80945fa39af250dc02000000, repr=<ray.serve._private.proxy.ProxyActor object at 0x7f7fe8e05310>)
File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 437, in result
return self.__get_result()
File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
raise self._exception
File "/usr/local/lib/python3.8/site-packages/ray/serve/_private/proxy.py", line 1202, in init
self.wrapped_http_proxy, **middleware.options
AttributeError: 'Middleware' object has no attribute 'options'

Reproduction script

requests.get(f"http://raycluster-kuberay-head-svc.kube-ray.svc.cluster.local:8000/classify?sentence={prompt}")
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 203, in _new_conn
sock = connection.create_connection(
File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 790, in urlopen
response = self._make_request(
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 496, in _make_request
conn.request(
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 395, in request
self.endheaders()
File "/usr/local/lib/python3.9/http/client.py", line 1274, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.9/http/client.py", line 1034, in _send_output
self.send(msg)
File "/usr/local/lib/python3.9/http/client.py", line 974, in send
self.connect()
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 243, in connect
self.sock = self._new_conn()
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 218, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f630e4e3f40>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 486, in send
resp = conn.urlopen(
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 844, in urlopen
retries = retries.increment(
File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 515, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='raycluster-kuberay-head-svc.kube-ray.svc.cluster.local', port=8000): Max retries exceeded with url: /classify?sentence=This%20was%20a%20masterpiece.%20Not%20completely%20faithful%20to%20the%20books,%20but%20enthralling%20from%20beginning%20to%20end.%20Might%20be%20my%20favorite%20of%20the%20three. (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f630e4e3f40>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 73, in get
return request("get", url, params=params, **kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 519, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='raycluster-kuberay-head-svc.kube-ray.svc.cluster.local', port=8000): Max retries exceeded with url: /classify?sentence=This%20was%20a%20masterpiece.%20Not%20completely%20faithful%20to%20the%20books,%20but%20enthralling%20from%20beginning%20to%20end.%20Might%20be%20my%20favorite%20of%20the%20three. (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f630e4e3f40>: Failed to establish a new connection: [Errno 111] Connection refused'))

Anything else

the example i am using is this:
https://docs.ray.io/en/latest/serve/tutorials/text-classification.html

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

[echzhai]

bash-4.4# serve status --address=http://raycluster-kuberay-head-svc.kube-ray.svc.cluster.local:8265
proxies:
de418112b1e7a736884003a619bbe8ab465bb9077a8cdff29e125f4e: STARTING
ef52d5aa58d16967c8c193509c9bd2658fe00e137c795e7f30b7142e: STARTING
applications:
default:
status: RUNNING
message: ''
last_deployed_time_s: 1705501369.2617495
deployments:
DistilBertModel:
status: HEALTHY
status_trigger: CONFIG_UPDATE_COMPLETED
replica_states: {}
message: ''
APIIngress:
status: HEALTHY
status_trigger: CONFIG_UPDATE_COMPLETED
replica_states:
RUNNING: 1
message: ''
target_capacity: null

[Yicheng-Lu-llll]

See this issue here: ray-project/ray#42370

[echzhai]

thank you. it works

[echzhai]

lib does not compatible issue.