.get_result() method of async endpoint response not working

[joaopcm1996]

Describe the bug
When calling get_result() on the output of a .predict_async() call, a method (_handle_response()) is called which is not implemented for an AsyncPredictor, resulting in an error.

To reproduce

from sagemaker.predictor_async import AsyncPredictor

predictor_async_wrapper = AsyncPredictor(predictor=predictor_async,
                                         name='async-pred')

async_response = predictor_async_wrapper.predict_async(data='This will get picked up by the endpoint.',initial_args={'Accept':'application/json'})

async_response.get_result()

Expected behavior
No errors, get response back

Screenshots or logs

AttributeError Traceback (most recent call last)

/tmp/ipykernel_28650/2526695841.py in <cell line: 1>()
----> 1 async_response.get_result()

~/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/sagemaker/async_inference/async_inference_response.py in get_result(self, waiter_config)
     72         if self._result is None:
     73             if waiter_config is None:
---> 74                 self._result = self._get_result_from_s3(self.output_path)
     75             else:
     76                 self._result = self.predictor_async._wait_for_output(

~/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/sagemaker/async_inference/async_inference_response.py in _get_result_from_s3(self, output_path)
     87         try:
     88             response = self.predictor_async.s3_client.get_object(Bucket=bucket, Key=key)
---> 89             return self.predictor_async.predictor._handle_response(response)
     90         except ClientError as ex:
     91             if ex.response["Error"]["Code"] == "NoSuchKey":

AttributeError: 'AsyncPredictor' object has no attribute '_handle_response'

System information
A description of your system. Please provide:

• **SageMaker Python SDK version: 2.110.0

[martinRenou]

Thank you for opening an issue.

It seems you have created an AsyncPredictor wrapping another AsyncPredictor. The AsyncPredictor must wrap a Predictor object instead.

What was the reason for wrapping an AsyncPredictor?

[joaopcm1996]

predictor_async is not actually an AsyncPredictor, it's just a normal Predictor

[martinRenou]

Thank you for answering. Would you be able to test something for me?

Could you print print(async_response.predictor_async) and print(async_response.predictor_async.predictor) just before your get_result() call? And show me what it prints.

The error you have is AttributeError: 'AsyncPredictor' object has no attribute '_handle_response' when reading the property predictor of the predictor_async property of the AsyncInferenceResponse object (what you called async_response). So it seems that async_response.predictor_async.predictor is an AsyncPredictor and not a normal Predictor.

[martinRenou]

Closing for triaging. Feel free to continue the discussion if you require more help though, happy to help you.