From db608332e71b7202dde31180fe9f543eb7e5cca7 Mon Sep 17 00:00:00 2001 From: Robert Metzger Date: Thu, 16 Apr 2015 09:36:33 +0200 Subject: [PATCH] [FLINK-1875] Add figure explaining slots and parallelism This closes #604 --- docs/config.md | 7 +- docs/faq.md | 5 +- docs/img/slots_parallelism.svg | 695 +++++++++++++++++++++++++++++++++ pom.xml | 3 +- 4 files changed, 702 insertions(+), 8 deletions(-) create mode 100644 docs/img/slots_parallelism.svg diff --git a/docs/config.md b/docs/config.md index 251ce03ad28df..1068152a06cd6 100644 --- a/docs/config.md +++ b/docs/config.md @@ -370,8 +370,7 @@ system, such as */tmp* in Linux systems. ### Configuring TaskManager processing slots -A processing slot allows Flink to execute a distributed DataSet transformation, such as a -data source or a map-transformation. +Flink executes a program in parallel by splitting it into subtasks and scheduling these subtasks to processing slots. Each Flink TaskManager provides processing slots in the cluster. The number of slots is typically proportional to the number of available CPU cores __of each__ TaskManager. @@ -383,6 +382,4 @@ The command line value therefore is called `-p` (for parallelism). In addition, to [set the number of slots in the programming APIs](programming_guide.html#parallel-execution) for the whole application and individual operators. -Flink is currently scheduling an application to slots by "filling" them up. -If the cluster has 20 machines with 2 slots each (40 slots in total) but the application is running -with a parallelism of 20, only 10 machines will process data. + \ No newline at end of file diff --git a/docs/faq.md b/docs/faq.md index 5c31bc66dc6be..c7794a14b34c2 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -103,8 +103,9 @@ parallelism has to be 1 and set it accordingly. The parallelism can be set in numerous ways to ensure a fine-grained control over the execution of a Flink program. See -[Configuration](config.html#common-options) for detailed instructions on how to -set the parallelism. +the [Configuration guide](config.html#common-options) for detailed instructions on how to +set the parallelism. Also check out [this figure](config.html#configuring-taskmanager-processing-slots) detailing +how the processing slots and parallelism are related to each other. ## Errors diff --git a/docs/img/slots_parallelism.svg b/docs/img/slots_parallelism.svg new file mode 100644 index 0000000000000..14e0960af366b --- /dev/null +++ b/docs/img/slots_parallelism.svg @@ -0,0 +1,695 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/pom.xml b/pom.xml index 4232fb7fc74da..eb73b3b842814 100644 --- a/pom.xml +++ b/pom.xml @@ -646,7 +646,7 @@ under the License. org.apache.rat apache-rat-plugin - 0.10 + 0.11 false @@ -697,6 +697,7 @@ under the License. **/*.log docs/**/bootstrap* + docs/img/*.svg **/resources/**/font-awesome/** **/resources/**/jquery* **/resources/**/bootstrap*