Skip to content
This repository was archived by the owner on Jul 7, 2023. It is now read-only.

Conversation

evalphobia
Copy link
Contributor

Problem

When we use t2t-query-server and TensorFlow Serving is hosted by our own(gRPC), the responses are from TensorFlow Serving converted to ndarray.

response = stub.Predict(request, timeout_secs)
outputs = tf.make_ndarray(response.outputs["outputs"])
scores = tf.make_ndarray(response.outputs["scores"])
assert len(outputs) == len(scores)
return [{ # pylint: disable=g-complex-comprehension
"outputs": output,
"scores": score
} for output, score in zip(outputs, scores)]

But when we use it with Cloud ML Engine, the responses are not converted.

prediction = api.projects().predict(body=input_data, name=parent).execute()
return prediction["predictions"]

then, got error,

Traceback (most recent call last):
  File "//anaconda3/envs/py27/lib/python2.7/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "//anaconda3/envs/py27/lib/python2.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "//anaconda3/envs/py27/lib/python2.7/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "//anaconda3/envs/py27/lib/python2.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "//anaconda3/envs/py27/lib/python2.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/evalphobia/repos/my-t2t-transformer-jp/export/query/src/server.py", line 101, in endpoint
    outputs = serving_utils.predict([inputs], problem, request_fn)
  File "//anaconda3/envs/py27/lib/python2.7/site-packages/tensor2tensor/serving/serving_utils.py", line 165, in predict
    for prediction in predictions
  File "//anaconda3/envs/py27/lib/python2.7/site-packages/tensor2tensor/serving/serving_utils.py", line 97, in _decode
    if len(output_ids.shape) > 1:
AttributeError: 'list' object has no attribute 'shape'

Changes

Just convert them to ndarray.

@googlebot
Copy link

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it!) and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

@googlebot googlebot added the cla: no PR author has not signed CLA label Sep 3, 2019
@evalphobia
Copy link
Contributor Author

@googlebot I signed it

@googlebot
Copy link

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

@googlebot googlebot added cla: yes PR author has signed CLA and removed cla: no PR author has not signed CLA labels Sep 3, 2019
Copy link
Contributor

@lukaszkaiser lukaszkaiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

@lukaszkaiser lukaszkaiser merged commit 39232f4 into tensorflow:master Sep 4, 2019
@lukaszkaiser
Copy link
Contributor

Great thanks for correcting this!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
cla: yes PR author has signed CLA
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants