Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support cloud events for KFServing custom framework #1159

Closed
yuzisun opened this issue Oct 26, 2020 · 4 comments · Fixed by #1343
Closed

Support cloud events for KFServing custom framework #1159

yuzisun opened this issue Oct 26, 2020 · 4 comments · Fixed by #1343

Comments

@yuzisun
Copy link
Member

yuzisun commented Oct 26, 2020

/kind feature

Describe the solution you'd like
As a user when doing feature transformation the payload can be protobuf/avro or some other binary format delivered via HTTP or KAFKA in cloudevent binary format(https://github.com/cloudevents/spec/blob/v1.0.1/kafka-protocol-binding.md#32-binary-content-mode) but currently KFServing custom framework only supports JSON.

  • I'd like to send cloud event to KFServer via KAFKA in following format and KFServer pass the decoded cloud event to preprocess or predict handler
------------------ Message -------------------
Topic Name: mytopic
------------------- key ----------------------
Key: mykey
------------------ headers -------------------
ce_specversion: "1.0"
ce_type: "com.example.someevent"
ce_source: "/mycontext/subcontext"
ce_id: "1234-1234-1234"
ce_time: "2018-04-05T03:56:24Z"
content-type: application/protobuf
------------------- value --------------------
      ... application data encoded in protobuf ...
-----------------------------------------------

             consume                       http(cloudevent)
KAFKA ----------> Kafka Event Source -----------------> InferenceService

def has_ce_headers(headers: typing.Dict[str, str]) -> bool:
    return (
        "ce-specversion" in headers
        and "ce-source" in headers
        and "ce-type" in headers
        and "ce-id" in headers
    )
    

 if has_ce_headers(self.request.headers):
        event = from_http(self.request.headers, self.request.body)
        body = event['data']
 else:
        try:
            body = json.loads(self.request.body)
        except json.decoder.JSONDecodeError as e:
            raise tornado.web.HTTPError(
                 status_code=HTTPStatus.BAD_REQUEST,
                 reason="Unrecognized request format: %s" % e)
  • Currently HTTP headers are not exposed to the preprocess or predict handler, if a cloud event is sent to KFServer it is not possible for user to check the http headers which can be useful for tracing and examining the original event source(kafka or http).
    async def predict(self, request: Dict, headers: Dict) -> Dict:
    async def preprocess(self, request: Dict, headers: Dict) -> Dict:

  • Also it would be nice if KFServing can generate cloud event response so that it works with knative eventing channels seamlessly.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

@issue-label-bot
Copy link

Issue-Label Bot is automatically applying the labels:

Label Probability
area/engprod 0.91

Please mark this comment with 👍 or 👎 to give our bot feedback!
Links: app homepage, dashboard and code for this bot.

1 similar comment
@issue-label-bot
Copy link

Issue-Label Bot is automatically applying the labels:

Label Probability
area/engprod 0.91

Please mark this comment with 👍 or 👎 to give our bot feedback!
Links: app homepage, dashboard and code for this bot.

@issue-label-bot
Copy link

Issue-Label Bot is automatically applying the labels:

Label Probability
area/front-end 0.85

Please mark this comment with 👍 or 👎 to give our bot feedback!
Links: app homepage, dashboard and code for this bot.

1 similar comment
@issue-label-bot
Copy link

Issue-Label Bot is automatically applying the labels:

Label Probability
area/front-end 0.85

Please mark this comment with 👍 or 👎 to give our bot feedback!
Links: app homepage, dashboard and code for this bot.

@yuzisun yuzisun changed the title Support cloud events/HTTP headers for KFServing custom framework Support cloud events for KFServing custom framework Dec 22, 2020
k8s-ci-robot pushed a commit that referenced this issue Feb 6, 2021
* parent a0a52f8
author Manasvi Tickoo <MTICKOO@bloomberg.net> 1611275174 -0500
committer Manasvi Tickoo <MTICKOO@bloomberg.net> 1612370131 -0500

1159 - cloudevent support ofr kfserving

PR comments

Fix clpudevent headers

Add unit test for cloud event messages

Add avro unit test

Add avro to kfserving requirements.txt

PR comments

kfserving check ce-contenttyp before unmarshalling

Update k8s libraries to 0.19.2 (#1305)

* go mod tidy

* Update k8s libs to 0.19.2, fix issues, and run make tests

* Handle ce to_binary return type wile building resp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants