Add ambassador annotation to tf-serving http proxy

[lluunn]

Verified visiting xxx/models/inception/ with browser (logged in) get response from http server:
Hello World

Related: #290

---

This change is 

[jlewi]

/assign @ankushagarwal

[ankushagarwal]

Can you update the relevant docs as well?

[ankushagarwal]

Have you tried running a POST request to the model, it is not showing up in the ambassador routing table

[lluunn]

Not yet, I was trying to figure out how to programmatically get auth for IAP so I can send POST request. I was thinking to add doc when I have that example ready -- #293

[ankushagarwal]

You don't need to go through IAP to test this. You can kubectl port-forward <ambassador-pod> 8000:80 and then you can hit localhost:8000 to access ambassador

[lluunn]

[lluunn]

Actually, POST is not working now

[lluunn]

Oh it's working, the route is correct, but just prediction failed.
Here is the logs from http-proxy:

E 180228 00:47:36 web:1590] Uncaught exception POST /model/inception:predict (10.0.0.7)
HTTPServerRequest(protocol='http', host='kubeflow.luffy.info', method='POST', uri='/model/inception:predict', version='HTTP/1.1', remote_ip='10.0.0.7', headers={'X-Request-Id': '8fa09935-447a-48b0-90cd-8a70cf6e2e1c', 'Via': '1.1 google', 'X-Goog-Iap-Jwt-Assertion': 'eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6InFWNU9IUSJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmdvb2dsZS5jb20vaWFwIiwic3ViIjoiYWNjb3VudHMuZ29vZ2xlLmNvbToxMDk2MzYxNDc3MjgxMTU1ODExODUiLCJlbWFpbCI6Imx1bmthaUBnb29nbGUuY29tIiwiYXVkIjoiL3Byb2plY3RzLzcxMTg5MDE2MDQzNC9nbG9iYWwvYmFja2VuZFNlcnZpY2VzLzY4OTgyMjI5NzU3NzA2NzA0MTQiLCJleHAiOjE1MTk3Nzk0NTQsImlhdCI6MTUxOTc3ODg1NCwiaGQiOiJnb29nbGUuY29tIn0.Cb1PHx9HhglXZgJVZg9Rr1ydm0fE3XGKk4SXP19aAC-nwbWGsf2IiooNhbVc-OYH2GzJsdPxDiVR9vkGhtyU1Q', 'X-Forwarded-For': '104.133.8.94, 35.186.239.86', 'X-Envoy-Expected-Rq-Timeout-Ms': '3000', 'Content-Length': '293440', 'Accept': '/', 'User-Agent': 'curl/7.52.1', 'X-Goog-Authenticated-User-Email': 'accounts.google.com:lunkai@google.com', 'Host': 'kubeflow.luffy.info', 'X-Forwarded-Proto': 'https', 'X-Cloud-Trace-Context': 'c039aff5d0c472e7e16ef4e76bb1a138/4893511970644272611', 'X-Envoy-Original-Path': '/models/inception/', 'Content-Type': 'application/x-www-form-urlencoded', 'X-Goog-Authenticated-User-Id': 'accounts.google.com:109636147728115581185'})
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/tornado/web.py", line 1511, in _execute
result = yield result
File "/usr/local/lib/python2.7/site-packages/tornado/gen.py", line 1055, in run
value = future.result()
File "/usr/local/lib/python2.7/site-packages/tornado/concurrent.py", line 238, in result
raise_exc_info(self._exc_info)
File "/usr/local/lib/python2.7/site-packages/tornado/gen.py", line 1063, in run
yielded = self.gen.throw(*exc_info)
File "/usr/src/app/server.py", line 116, in post
result = yield fwrap(stub.Predict.future(request, self.settings['rpc_timeout']))
File "/usr/local/lib/python2.7/site-packages/tornado/gen.py", line 1055, in run
value = future.result()
File "/usr/local/lib/python2.7/site-packages/tornado/concurrent.py", line 238, in result
raise_exc_info(self._exc_info)
File "", line 3, in raise_exc_info
AbortionError: AbortionError(code=StatusCode.INVALID_ARGUMENT, details="input tensor alias not found in signature: image_contents. Inputs expected to be in the set {inputs}.")
[E 180228 00:47:36 web:2063] 500 POST /model/inception:predict (10.0.0.7) 666.82ms

[ankushagarwal]

Can you create an issue to track this issue in http-proxy?

The change to tf-serving.libsonnet looks good, can you update docs on how to use the newly added routes (without IAP) by talking to ambassador service?

[lluunn]

Updated the README.
The http prediction is a model/input problem, will have a working example for #293

[ankushagarwal]

/lgtm

[googlebot]

So there's good news and bad news.

👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there.

😕 The bad news is that it appears that one or more commits were authored by someone other than the pull request submitter. We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that here in the pull request.

Note to project maintainer: This is a terminal state, meaning the cla/google commit status will not change from this State. It's up to you to confirm consent of the commit author(s) and merge this pull request when appropriate.

[jlewi]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ankushagarwal, jlewi
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:

Assign the PR to them by writing /assign in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment