New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FLINK-21932] Add Python SDK showcase #7
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Thanks @igalshilman.
I just have some minor typo findings that I'll address while merging.
## Tutorial Sections | ||
|
||
The [__main__.py](showcase/__main__.py) file demonstrates SDK concepts at length, and highly recommend | ||
to read trough it. The sections below are copied from that file with some of the comments removed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
type: trough
-> through
|
||
## Tutorial Sections | ||
|
||
The [__main__.py](showcase/__main__.py) file demonstrates SDK concepts at length, and highly recommend |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The [__main__.py](showcase/__main__.py) file demonstrates SDK concepts at length, and highly recommend | |
The [__main__.py](showcase/__main__.py) file demonstrates SDK concepts at length, and it is highly recommended |
### Serving | ||
|
||
* First, lets define a simple function that computes personalized greetings messages based on the number of times | ||
That this function was invoked. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That this function was invoked. | |
that this function was invoked. |
|
||
* Exposing the handler via HTTP | ||
The code below handles the physical HTTP serving. | ||
In this case we chose aiohttp, although any other HTTP serving framework will do. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In this case we chose aiohttp, although any other HTTP serving framework will do. | |
In this case we chose `aiohttp`, although any other HTTP serving framework will do. |
# You can also define your own types using the type system, such as a JSON message ... | ||
val = message.as_type(GREET_JSON_TYPE) | ||
elif message.is_type(USER_PROFILE_PROTOBUF_TYPE): | ||
# Or protobuf |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
# Or protobuf | |
# Or Protobuf |
return str.encode('utf-8') | ||
|
||
|
||
GREET_JSON_TYPE = simple_type(typename="showcase/GreetRequest", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can change this to make_json_type
now.
copied from README.md
StateFun Python SDK Showcase
This project is intended for new StateFun users that would like to start implementing their StateFun application functions using Python.
The tutorial is streamlined and split into a few parts which we recommend to go through a specific order, as lay out below.
Each part is demonstrated with some code snippets plus comments to guide you through the SDK fundamentals.
Prerequisites
Building the example
Using venv
Tutorial Sections
The main.py file demonstrates SDK concepts at length, and highly recommend
to read trough it. The sections below are copied from that file with some of the comments removed.
Type System
This function demonstrates StateFun's type system using the Python SDK.
Messaging Primitives
This function demonstrates how to send messages to other functions.
Sending messages to egresses
To let your StateFun application interact with the outside world, functions may write messages
to egresses. This function demonstrates sending messages to an Apache Kafka or AWS Kinesis
egress, which is currently our most commonly used egresses that are natively supported by
StateFun.
Function state storage
Asynchronous operations
This function demonstrates performing asynchronous operations during a function invocation. It
is a common scenario for functions to have external dependencies in order for it to complete its
work, such as fetching enrichment information from remote databases.
Serving
That this function was invoked.
For demonstration purposes, this function prints to console the generated greetings
messages.
decode invocation requests dispatched from StateFun cluster, dispatch to the correct function,
and encode side-effects (e.g. storage updates, or invoking other functions)
as responses to be handled by StateFun.
The code below handles the physical HTTP serving.
In this case we chose aiohttp, although any other HTTP serving framework will do.
The only thing that the HTTP handler needs to do, is to pass the raw (bytes) request body
to StateFun's handler (defined above), and return the raw (bytes) StateFun's handler provided as a response.
To actually start serving run from one terminal:
python3 -m showcase
And from another:
docker-compose up
Next Steps
The setup you executed in the last part of this tutorial is not how you'd normally deploy StateFun processes
and functions. It's a rather simplified setup to allow you to explore the interaction between
functions and the StateFun processes by setting debugger breakpoints in the function code in your IDE.
We recommend now to take a look at a slightly more realistic setup, using Docker Compose, in the
Greeter Docker Compose Example.