[FLINK-18979][docs] Update docs to better emphasize remote functions #132

sjwiesman · 2020-08-17T19:23:16Z

No description provided.

sjwiesman · 2020-08-17T20:15:56Z

docs/index.md

-**Stateful Functions** is an open source framework that reduces the complexity of building and orchestrating distributed stateful applications at scale.
-It brings together the benefits of stream processing with Apache Flink® and Function-as-a-Service (FaaS) to provide a powerful abstraction for the next generation of event-driven architectures.
+**Stateful Functions** is a framework, developed under the umbrella of Apache Flink, that makes it simple to build **consistent stateful serverless applications**.
+It is designed to work with modern architectures, like stateless Kubernetes deployments or popular event-driven FaaS platforms like AWS Lambda, KNative, etc.,


Suggested change

It is designed to work with modern architectures, like stateless Kubernetes deployments or popular event-driven FaaS platforms like AWS Lambda, KNative, etc.,

It is designed to work with modern architectures, like cloud-native deployments or popular event-driven FaaS platforms like AWS Lambda, KNative, etc.,

This was my other idea for this sentence, I'd like to hear what people like more

Stateless k8s deployments, IMO, would probably catch people a bit confused from a first read.
Overall, I always had the feeling that due to how the project is named "Stateful Functions", every time we use the term "stateless" we are in a bit of risk of misguiding readers who don't fully understand the context the term is used in.

Agree with Gordon here. What about stating the problem this solves, instead? Or, briefly mentioning why state is such an issue in this kind of applications.

sjwiesman · 2020-08-17T20:17:51Z

docs/index.md

@@ -23,16 +23,16 @@ specific language governing permissions and limitations
 under the License.
 -->

-**Stateful Functions** is an open source framework that reduces the complexity of building and orchestrating distributed stateful applications at scale.
-It brings together the benefits of stream processing with Apache Flink® and Function-as-a-Service (FaaS) to provide a powerful abstraction for the next generation of event-driven architectures.
+**Stateful Functions** is a framework, developed under the umbrella of Apache Flink, that makes it simple to build **consistent stateful serverless applications**.


Suggested change

**Stateful Functions** is a framework, developed under the umbrella of Apache Flink, that makes it simple to build **consistent stateful serverless applications**.

**Stateful Functions** is a framework, developed under the umbrella of Apache Flink, that simplifies building distributed stateful applications.

Other possible first sentence

I personally like the first version more, and have been using that tag line a lot in recent talks / current work-in-progress blog posts. However, I'm also not sure if the first version is "down-scoping" the project as well.

I'd leave this to @igalshilman / @StephanEwen though, as it seems like more a question of how we want the project to be pitched moving forward.

Just my two more cents:
From how I read the changes in the PR, it seems like the overall goal is to make remote functions / the "Flink as DB" mode more prominent and the main focus of the project.

In that sense, I would vote for **... makes it simple to build consistent stateful serverless applications as that seems to be more coherent with the goal.

I think here we're going from concepts people can relate to to a mouthful of buzzwords, to be honest. "Stateful serverless" is good to create hype for conferences and such, but I don't think the people who might use StateFun will relate to that more than to "this is like if you bring what's good in stateful stream processing and FaaS together".

Also, I wouldn't break the main sentence; the Flink part could be attached at the end, so you get an uninterrupted definition of what StateFun is.

"Stateful Functions is a framework that makes it simple to build consistent stateful serverless applications, developed under the umbrella of Apache Flink."

tzulitai · 2020-08-18T02:30:05Z

While you're at it, once we've reached consensus of the updated contents of the homepage, could you update the README.mds (at root and under statefun-python-sdk) as well?

morsapaes · 2020-08-18T08:52:17Z

docs/index.md

@@ -23,16 +23,16 @@ specific language governing permissions and limitations
 under the License.
 -->

-**Stateful Functions** is an open source framework that reduces the complexity of building and orchestrating distributed stateful applications at scale.
-It brings together the benefits of stream processing with Apache Flink® and Function-as-a-Service (FaaS) to provide a powerful abstraction for the next generation of event-driven architectures.
+**Stateful Functions** is a framework, developed under the umbrella of Apache Flink, that makes it simple to build **consistent stateful serverless applications**.


I think here we're going from concepts people can relate to to a mouthful of buzzwords, to be honest. "Stateful serverless" is good to create hype for conferences and such, but I don't think the people who might use StateFun will relate to that more than to "this is like if you bring what's good in stateful stream processing and FaaS together".

morsapaes · 2020-08-18T08:54:06Z

docs/index.md

@@ -23,16 +23,16 @@ specific language governing permissions and limitations
 under the License.
 -->

-**Stateful Functions** is an open source framework that reduces the complexity of building and orchestrating distributed stateful applications at scale.
-It brings together the benefits of stream processing with Apache Flink® and Function-as-a-Service (FaaS) to provide a powerful abstraction for the next generation of event-driven architectures.
+**Stateful Functions** is a framework, developed under the umbrella of Apache Flink, that makes it simple to build **consistent stateful serverless applications**.


Also, I wouldn't break the main sentence; the Flink part could be attached at the end, so you get an uninterrupted definition of what StateFun is.

"Stateful Functions is a framework that makes it simple to build consistent stateful serverless applications, developed under the umbrella of Apache Flink."

morsapaes · 2020-08-18T11:50:16Z

docs/index.md

-**Stateful Functions** is an open source framework that reduces the complexity of building and orchestrating distributed stateful applications at scale.
-It brings together the benefits of stream processing with Apache Flink® and Function-as-a-Service (FaaS) to provide a powerful abstraction for the next generation of event-driven architectures.
+**Stateful Functions** is a framework, developed under the umbrella of Apache Flink, that makes it simple to build **consistent stateful serverless applications**.
+It is designed to work with modern architectures, like stateless Kubernetes deployments or popular event-driven FaaS platforms like AWS Lambda, KNative, etc.,


Agree with Gordon here. What about stating the problem this solves, instead? Or, briefly mentioning why state is such an issue in this kind of applications.

morsapaes · 2020-08-18T11:51:06Z

docs/index.md

-Which one is the best for you depends on your goals and prior experience.
-Whether you prefer a more theoretical or a practical approach, we hope you’ll find this section helpful.
+It brings together the benefits of Apache Flink®, streaming semantics for processing large datasets with low latency and bounded resource constraints,
+along with a runtime for modeling stateful entities that supports location transparency, concurrency, scaling, and resiliency. 


Suggested change

along with a runtime for modeling stateful entities that supports location transparency, concurrency, scaling, and resiliency.

along with a runtime for modeling stateful entities that supports location transparency, concurrency, scaling, and resilience.

-1 I think its "resiliency"

morsapaes · 2020-08-18T14:20:05Z

If we're changing the docs, should we also change the webpage in the Flink website? So that things are coherent.

sjwiesman · 2020-08-18T14:44:06Z

I've pushed an update to the home page. @tzulitai Yes I will update the README's after we reach consensus.

sjwiesman · 2020-08-18T14:44:28Z

@morsapaes yes, we can as a follow up.

sjwiesman · 2020-08-20T17:37:58Z

@tzulitai / @morsapaes when you get a chance I've updated the homepage based on the above discussion.

morsapaes

Looks really good, @sjwiesman! Left minor suggestions to remove some small inconsistencies.

morsapaes · 2020-08-21T09:50:51Z

docs/index.md

-**Stateful Functions** is an open source framework that reduces the complexity of building and orchestrating distributed stateful applications at scale.
-It brings together the benefits of stream processing with Apache Flink® and Function-as-a-Service (FaaS) to provide a powerful abstraction for the next generation of event-driven architectures.
+Stateful Functions is an API that simplifies the building of **distributed stateful applications** with a **runtime built for serverless architectures**.
+It brings together the benefits of stateful stream processing, the processing of large datasets with low latency and bounded resource constraints,


This confuses me a bit because of punctuation (🙄). Is "the processing of large datasets with low latency and bounded resource constraints" the unfold of "the benefits of stateful stream processing" or do you mean it as a separate "benefit"?

Other than that, I really like the rephrasing!

Hmm, yeah I'll try something

morsapaes · 2020-08-21T09:53:55Z

docs/sdk/index.md

@@ -25,3 +25,109 @@ KIND, either express or implied.  See the License for the
 specific language governing permissions and limitations
 under the License.
 -->
+
+Stateful Functions applications are composed of one or more modules.
+A module is a bundle of functions that are loaded by the runtime and available to be messaged.


Suggested change

A module is a bundle of functions that are loaded by the runtime and available to be messaged.

A module is a bundle of functions that are loaded by the runtime and become available to be messaged.

morsapaes · 2020-08-21T09:54:32Z

docs/sdk/index.md

+A module is a bundle of functions that are loaded by the runtime and available to be messaged.
+Functions from all loaded modules are multiplexed and free to message each other arbitrarily.
+
+Stateful Functions supports two types of modules: Embedded and Remote.


Suggested change

Stateful Functions supports two types of modules: Embedded and Remote.

Stateful Functions supports two types of modules: Remote and Embedded.

To keep up with the order you used below.

morsapaes · 2020-08-21T09:56:53Z

docs/sdk/index.md

+``meta`` contains auxillary information about the module.
+The ``spec`` describes the functions contained within the module and defines their persisted values.


Suggested change

``meta`` contains auxillary information about the module.

The ``spec`` describes the functions contained within the module and defines their persisted values.

The ``meta`` section contains auxiliary information about the module; while ``spec`` describes the functions contained within the module and defines their persisted values.

morsapaes · 2020-08-21T09:57:24Z

docs/sdk/index.md

+### Defining Functions
+
+``module.spec.functions`` declares a list of ``function`` objects that are implemented by the remote module.
+A ``function`` is described via a number of properties.


Suggested change

A ``function`` is described via a number of properties.

A ``function`` is described via a number of properties:

morsapaes · 2020-08-21T10:22:56Z

docs/sdk/index.md

+
+* ``function.meta.kind``
+    * The protocol used to communicate with the remote function.
+    * Supported Values - ``http``


Suggested change

* Supported Values - ``http``

* Supported values: ``http``

Making these consistent (more below).

morsapaes · 2020-08-21T10:23:26Z

docs/sdk/index.md

+    * The function type, defined as ``<namespace>/<name>``.
+* ``function.spec.endpoint``
+    * The endpoint at which the function is reachable.
+    * Supported schemes are: ``http``, ``https``.


Suggested change

* Supported schemes are: ``http``, ``https``.

* Supported schemes: ``http``, ``https``.

morsapaes · 2020-08-21T10:23:48Z

docs/sdk/index.md

+    * Default for `expireAfter` - 0, meaning that state expiration is disabled.
+* ``function.spec.maxNumBatchRequests``
+    * The maximum number of records that can be processed by a function for a particular ``address`` before invoking backpressure on the system.
+    * Default - 1000


Suggested change

* Default - 1000

* Default: 1000

morsapaes · 2020-08-21T10:23:56Z

docs/sdk/index.md

+    * Default - 1000
+* ``function.spec.timeout``
+    * The maximum amount of time for the runtime to wait for the remote function to return before failing.
+    * Default - 1 min


Suggested change

* Default - 1 min

* Default: 1 min

morsapaes · 2020-08-21T10:27:20Z

docs/sdk/index.md

+This module type only supports JVM based languages and are defined by implementing the ``StatefulFunctionModule`` interface.
+Embedded modules offer a single configuration method where stateful functions bind to the system based on their [function type]({{ site.baseurl }}/concepts/logical.html#function-address).
+Runtime configurations are available through the ``globalConfiguration``, which is the union of all configurations in the applications ``flink-conf.yaml`` under the prefix ``statefun.module.global-config`` and any command line arguments passed in the form ``--key value``.


Suggested change

This module type only supports JVM based languages and are defined by implementing the ``StatefulFunctionModule`` interface.

Embedded modules offer a single configuration method where stateful functions bind to the system based on their [function type]({{ site.baseurl }}/concepts/logical.html#function-address).

Runtime configurations are available through the ``globalConfiguration``, which is the union of all configurations in the applications ``flink-conf.yaml`` under the prefix ``statefun.module.global-config`` and any command line arguments passed in the form ``--key value``.

This module type only supports JVM-based languages and is defined by implementing the ``StatefulFunctionModule`` interface.

Embedded modules offer a single configuration method where stateful functions bind to the system based on their [function type]({{ site.baseurl }}/concepts/logical.html#function-address).

Runtime configurations are available through the ``globalConfiguration``, which is the union of all configurations in the applications ``flink-conf.yaml`` under the prefix ``statefun.module.global-config``, and any command line arguments passed in the form ``--key value``.

…n top

sjwiesman · 2020-08-21T18:36:19Z

Thanks for the review @morsapaes . I've addressed your feedback and also updated the README's in a seperate commit. I'm going on vacation next week so I went ahead and squashed my fixup commits.

tzulitai · 2020-08-31T05:54:38Z

LGTM, merging. Thanks @sjwiesman!

sjwiesman commented Aug 17, 2020

View reviewed changes

morsapaes reviewed Aug 18, 2020

View reviewed changes

morsapaes reviewed Aug 21, 2020

View reviewed changes

sjwiesman added 5 commits August 21, 2020 13:34

[FLINK-18979][docs] Move modules to overview and put remote modules o…

be0f65e

…n top

[FLINK-18979][docs] Move Python SDK above Java SDK

d3ca2f5

[FLINK-18979][docs] Default to remote modules in connectors

d1a14cc

[FLINK-18979][docs] Update homepage introduction

5e61add

[FLINK-18979][docs] Update READMEs

1be4371

sjwiesman force-pushed the remote-modules branch from 55e5631 to 1be4371 Compare August 21, 2020 18:34

tzulitai closed this in c483184 Aug 31, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FLINK-18979][docs] Update docs to better emphasize remote functions #132

[FLINK-18979][docs] Update docs to better emphasize remote functions #132

sjwiesman commented Aug 17, 2020

sjwiesman Aug 17, 2020

tzulitai Aug 18, 2020

morsapaes Aug 18, 2020

sjwiesman Aug 17, 2020

tzulitai Aug 18, 2020

tzulitai Aug 18, 2020

morsapaes Aug 18, 2020

morsapaes Aug 18, 2020

tzulitai commented Aug 18, 2020

morsapaes Aug 18, 2020

morsapaes Aug 18, 2020

morsapaes Aug 18, 2020

morsapaes Aug 18, 2020

sjwiesman Aug 18, 2020

morsapaes commented Aug 18, 2020 •

edited

sjwiesman commented Aug 18, 2020

sjwiesman commented Aug 18, 2020

sjwiesman commented Aug 20, 2020

morsapaes left a comment

morsapaes Aug 21, 2020

sjwiesman Aug 21, 2020

morsapaes Aug 21, 2020

morsapaes Aug 21, 2020

morsapaes Aug 21, 2020

morsapaes Aug 21, 2020

morsapaes Aug 21, 2020

morsapaes Aug 21, 2020

morsapaes Aug 21, 2020

morsapaes Aug 21, 2020

morsapaes Aug 21, 2020

morsapaes Aug 21, 2020

morsapaes Aug 21, 2020

sjwiesman commented Aug 21, 2020

tzulitai commented Aug 31, 2020

	It is designed to work with modern architectures, like stateless Kubernetes deployments or popular event-driven FaaS platforms like AWS Lambda, KNative, etc.,
	It is designed to work with modern architectures, like cloud-native deployments or popular event-driven FaaS platforms like AWS Lambda, KNative, etc.,

	Stateful Functions is a framework, developed under the umbrella of Apache Flink, that makes it simple to build consistent stateful serverless applications.
	Stateful Functions is a framework, developed under the umbrella of Apache Flink, that simplifies building distributed stateful applications.

	along with a runtime for modeling stateful entities that supports location transparency, concurrency, scaling, and resiliency.
	along with a runtime for modeling stateful entities that supports location transparency, concurrency, scaling, and resilience.

	A module is a bundle of functions that are loaded by the runtime and available to be messaged.
	A module is a bundle of functions that are loaded by the runtime and become available to be messaged.

	Stateful Functions supports two types of modules: Embedded and Remote.
	Stateful Functions supports two types of modules: Remote and Embedded.

		``meta`` contains auxillary information about the module.
		The ``spec`` describes the functions contained within the module and defines their persisted values.

	``meta`` contains auxillary information about the module.
	The ``spec`` describes the functions contained within the module and defines their persisted values.
	The ``meta`` section contains auxiliary information about the module; while ``spec`` describes the functions contained within the module and defines their persisted values.

	A ``function`` is described via a number of properties.
	A ``function`` is described via a number of properties:

	* Supported schemes are: ``http``, ``https``.
	* Supported schemes: ``http``, ``https``.

[FLINK-18979][docs] Update docs to better emphasize remote functions #132

[FLINK-18979][docs] Update docs to better emphasize remote functions #132

Conversation

sjwiesman commented Aug 17, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tzulitai commented Aug 18, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

morsapaes commented Aug 18, 2020 • edited

sjwiesman commented Aug 18, 2020

sjwiesman commented Aug 18, 2020

sjwiesman commented Aug 20, 2020

morsapaes left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sjwiesman commented Aug 21, 2020

tzulitai commented Aug 31, 2020

morsapaes commented Aug 18, 2020 •

edited