Keras code example for TF serving

[dimitreOliveira]

Hey there, I would be happy to write a Keras code example showcasing how to server Keras models using TF serving, let me know if you think that this would be useful, if yes it would be awesome if you can mention if you like to see something specific like serving vision, text, multiple models, or if just a minimal example would be fine.

[tilakrayal]

@dimitreOliveira,
Could you please take a look at this official doc link and confirm if you are looking for the similar information? Thank you!

[dimitreOliveira]

Hi @tilakrayal , thanks for the reference, but I am actually offering to write a "Keras code example" on how to do that with different use cases that might be helpful for new users, what I would like to know is if the Keras team thinks that this could be a good addition or if it wouldn't add to much here.

[gowthamkpr]

@dimitreOliveira Can you please let us know what usecases are you proposing? Thanks!!

[dimitreOliveira]

Hi @gowthamkpr ,

Since the code examples are usually small I have two ideas of how it could go:

• Option 1: A two parts example where first I talk about TFServing, some best practices, etc and show how to serve a simple model (could be a vision model), in the second part I show how to create a custom signature that allows the user to embed the preprocessing and postprocessing logic and then serve the model with it, this is to showcase some of the advantages of TFServing.
• Option 2: The code example would be a collection of different types of models being served with TFServing, vision model, text model, structured data model, maybe a GAN or some forest mode, could be anything just to show its flexibility.

Do you think that any of those ideas would turn into good code examples?

[innat]

It would be great fit for https://keras.io/examples/keras_recipes/
cc. @fchollet

[haifeng-jin]

Does these all fit into a Colab notebook?

[haifeng-jin]

https://www.tensorflow.org/tfx/tutorials/serving/rest_simple
@dimitreOliveira
Will the contributed tutorial different from this one?

[dimitreOliveira]

Hi @haifeng-jin , yes option 1 would be similar to that with some additional information about model signatures, I believe that it could fit well into a notebook. Another advantage is that we would have an example of that on the Keras code example page, it could be easier for new users to find it. Let me know if you think that it would be a good addition.

[PierrickPochelu]

Why use keras if you can use more dedicated inference frameworks such as tensorrt, openvino, onnx-runtime, TF ...

[dimitreOliveira]

I am not quite sure of what you mean here, but my idea is to create an example of serving Keras models with TFServing, both of them belong to the TensorFlow ecosystem.

[haifeng-jin]

@dimitreOliveira
Thank you!
Wait for @fchollet comment before you start.

[fchollet]

Hi Dimitre,

Thanks for offering to write an example on TFServing + Keras, I think it would be valuable. You can just open a PR on https://github.com/keras-team/keras-io. I agree that option 1 (focus on best practices) is more helpful.

[dimitreOliveira]

Awesome, thank you very much guys, I will start working on this next week.