Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FLINK-15684][docs] Add taskmanager.memory.flink.size to common options #10916

Closed
wants to merge 2 commits into from
Closed

[FLINK-15684][docs] Add taskmanager.memory.flink.size to common options #10916

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
20a066e
[FLINK-15684][docs] Add taskmanager.memory.flink.size to common options
dawidwys Jan 21, 2020
dc81015
[FLINK-15684][docs] Regenerate documentation
dawidwys Jan 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 7 additions & 1 deletion docs/_includes/generated/common_section.html
View file
Open in desktop
Expand Up @@ -14,11 +14,17 @@
<td>String</td>
<td>JVM heap size for the JobManager.</td>
</tr>
<tr>
<td><h5>taskmanager.memory.flink.size</h5></td>
<td style="word-wrap: break-word;">(none)</td>
<td>MemorySize</td>
<td>Total Flink Memory size for the TaskExecutors. This includes all the memory that a TaskExecutor consumes, except for JVM Metaspace and JVM Overhead. It consists of Framework Heap Memory, Task Heap Memory, Task Off-Heap Memory, Managed Memory, and Network Memory. See also 'taskmanager.memory.process.size' for total process memory size configuration.</td>
</tr>
<tr>
<td><h5>taskmanager.memory.process.size</h5></td>
<td style="word-wrap: break-word;">(none)</td>
<td>MemorySize</td>
<td>Total Process Memory size for the TaskExecutors. This includes all the memory that a TaskExecutor consumes, consisting of Total Flink Memory, JVM Metaspace, and JVM Overhead. On containerized setups, this should be set to the container memory.</td>
<td>Total Process Memory size for the TaskExecutors. This includes all the memory that a TaskExecutor consumes, consisting of Total Flink Memory (configured with 'taskmanager.memory.flink.size'), JVM Metaspace, and JVM Overhead. On containerized setups, this should be set to the container memory.</td>
</tr>
<tr>
<td><h5>parallelism.default</h5></td>
Expand Down
4 changes: 2 additions & 2 deletions docs/_includes/generated/task_manager_memory_configuration.html
View file
Open in desktop
Expand Up @@ -12,7 +12,7 @@
<td><h5>taskmanager.memory.flink.size</h5></td>
<td style="word-wrap: break-word;">(none)</td>
<td>MemorySize</td>
<td>Total Flink Memory size for the TaskExecutors. This includes all the memory that a TaskExecutor consumes, except for JVM Metaspace and JVM Overhead. It consists of Framework Heap Memory, Task Heap Memory, Task Off-Heap Memory, Managed Memory, and Network Memory.</td>
<td>Total Flink Memory size for the TaskExecutors. This includes all the memory that a TaskExecutor consumes, except for JVM Metaspace and JVM Overhead. It consists of Framework Heap Memory, Task Heap Memory, Task Off-Heap Memory, Managed Memory, and Network Memory. See also 'taskmanager.memory.process.size' for total process memory size configuration.</td>
</tr>
<tr>
<td><h5>taskmanager.memory.framework.heap.size</h5></td>
Expand Down Expand Up @@ -84,7 +84,7 @@
<td><h5>taskmanager.memory.process.size</h5></td>
<td style="word-wrap: break-word;">(none)</td>
<td>MemorySize</td>
<td>Total Process Memory size for the TaskExecutors. This includes all the memory that a TaskExecutor consumes, consisting of Total Flink Memory, JVM Metaspace, and JVM Overhead. On containerized setups, this should be set to the container memory.</td>
<td>Total Process Memory size for the TaskExecutors. This includes all the memory that a TaskExecutor consumes, consisting of Total Flink Memory (configured with 'taskmanager.memory.flink.size'), JVM Metaspace, and JVM Overhead. On containerized setups, this should be set to the container memory.</td>
</tr>
<tr>
<td><h5>taskmanager.memory.segment-size</h5></td>
Expand Down
21 changes: 13 additions & 8 deletions flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
View file
Open in desktop
Expand Up @@ -257,20 +257,25 @@ public class TaskManagerOptions {
key("taskmanager.memory.process.size")
.memoryType()
.noDefaultValue()
.withDescription("Total Process Memory size for the TaskExecutors. This includes all the memory that a"
+ " TaskExecutor consumes, consisting of Total Flink Memory, JVM Metaspace, and JVM Overhead. On"
+ " containerized setups, this should be set to the container memory.");
.withDescription("Total Process Memory size for the TaskExecutors. This includes all the memory that a "
+ "TaskExecutor consumes, consisting of Total Flink Memory (configured with "
+ "'taskmanager.memory.flink.size'), JVM Metaspace, and JVM Overhead. On "
+ "containerized setups, this should be set to the container memory."
Comment on lines +260 to +263
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding the parenthesis with the reference to flink.size somehow reads as if one needs to configure this in order to make process.size work. I would rather suggest to add a sentence See also taskmanager.memory.flink.size for flink process mmeory size configuration.

);

/**
* Total Flink Memory size for the TaskExecutors.
*/
@Documentation.CommonOption(position = Documentation.CommonOption.POSITION_MEMORY)
public static final ConfigOption<MemorySize> TOTAL_FLINK_MEMORY =
key("taskmanager.memory.flink.size")
.memoryType()
.noDefaultValue()
.withDescription("Total Flink Memory size for the TaskExecutors. This includes all the memory that a"
+ " TaskExecutor consumes, except for JVM Metaspace and JVM Overhead. It consists of Framework Heap Memory,"
+ " Task Heap Memory, Task Off-Heap Memory, Managed Memory, and Network Memory.");
.memoryType()
.noDefaultValue()
.withDescription(String.format("Total Flink Memory size for the TaskExecutors. This includes all the "
+ "memory that a TaskExecutor consumes, except for JVM Metaspace and JVM Overhead. It consists of "
+ "Framework Heap Memory, Task Heap Memory, Task Off-Heap Memory, Managed Memory, and Network "
+ "Memory. See also '%s' for total process memory size configuration.",
TOTAL_PROCESS_MEMORY.key()));

/**
* Framework Heap Memory size for TaskExecutors.
Expand Down