Skip to content

[FLINK-15684][docs][bp-1.10] Add taskmanager.memory.flink.size to common options #10918

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dawidwys wants to merge 2 commits into from
Closed

[FLINK-15684][docs][bp-1.10] Add taskmanager.memory.flink.size to common options #10918

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
46a5a58
[FLINK-15684][docs] Add taskmanager.memory.flink.size to common options
dawidwys Jan 21, 2020
d318d59
[FLINK-15684][docs] Regenerate documentation
dawidwys Jan 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion docs/_includes/generated/common_section.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,17 @@
<td>String</td>
<td>JVM heap size for the JobManager.</td>
</tr>
<tr>
<td><h5>taskmanager.memory.flink.size</h5></td>
<td style="word-wrap: break-word;">(none)</td>
<td>MemorySize</td>
<td>Total Flink Memory size for the TaskExecutors. This includes all the memory that a TaskExecutor consumes, except for JVM Metaspace and JVM Overhead. It consists of Framework Heap Memory, Task Heap Memory, Task Off-Heap Memory, Managed Memory, and Network Memory. See also 'taskmanager.memory.process.size' for total process memory size configuration.</td>
</tr>
<tr>
<td><h5>taskmanager.memory.process.size</h5></td>
<td style="word-wrap: break-word;">(none)</td>
<td>MemorySize</td>
<td>Total Process Memory size for the TaskExecutors. This includes all the memory that a TaskExecutor consumes, consisting of Total Flink Memory, JVM Metaspace, and JVM Overhead. On containerized setups, this should be set to the container memory.</td>
<td>Total Process Memory size for the TaskExecutors. This includes all the memory that a TaskExecutor consumes, consisting of Total Flink Memory (configured with 'taskmanager.memory.flink.size'), JVM Metaspace, and JVM Overhead. On containerized setups, this should be set to the container memory.</td>
</tr>
<tr>
<td><h5>parallelism.default</h5></td>
Expand Down
4 changes: 2 additions & 2 deletions docs/_includes/generated/task_manager_memory_configuration.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
<td><h5>taskmanager.memory.flink.size</h5></td>
<td style="word-wrap: break-word;">(none)</td>
<td>MemorySize</td>
<td>Total Flink Memory size for the TaskExecutors. This includes all the memory that a TaskExecutor consumes, except for JVM Metaspace and JVM Overhead. It consists of Framework Heap Memory, Task Heap Memory, Task Off-Heap Memory, Managed Memory, and Network Memory.</td>
<td>Total Flink Memory size for the TaskExecutors. This includes all the memory that a TaskExecutor consumes, except for JVM Metaspace and JVM Overhead. It consists of Framework Heap Memory, Task Heap Memory, Task Off-Heap Memory, Managed Memory, and Network Memory. See also 'taskmanager.memory.process.size' for total process memory size configuration.</td>
</tr>
<tr>
<td><h5>taskmanager.memory.framework.heap.size</h5></td>
Expand Down Expand Up @@ -84,7 +84,7 @@
<td><h5>taskmanager.memory.process.size</h5></td>
<td style="word-wrap: break-word;">(none)</td>
<td>MemorySize</td>
<td>Total Process Memory size for the TaskExecutors. This includes all the memory that a TaskExecutor consumes, consisting of Total Flink Memory, JVM Metaspace, and JVM Overhead. On containerized setups, this should be set to the container memory.</td>
<td>Total Process Memory size for the TaskExecutors. This includes all the memory that a TaskExecutor consumes, consisting of Total Flink Memory (configured with 'taskmanager.memory.flink.size'), JVM Metaspace, and JVM Overhead. On containerized setups, this should be set to the container memory.</td>
</tr>
<tr>
<td><h5>taskmanager.memory.segment-size</h5></td>
Expand Down
21 changes: 13 additions & 8 deletions flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -257,20 +257,25 @@ public class TaskManagerOptions {
key("taskmanager.memory.process.size")
.memoryType()
.noDefaultValue()
.withDescription("Total Process Memory size for the TaskExecutors. This includes all the memory that a"
+ " TaskExecutor consumes, consisting of Total Flink Memory, JVM Metaspace, and JVM Overhead. On"
+ " containerized setups, this should be set to the container memory.");
.withDescription("Total Process Memory size for the TaskExecutors. This includes all the memory that a "
+ "TaskExecutor consumes, consisting of Total Flink Memory (configured with "
+ "'taskmanager.memory.flink.size'), JVM Metaspace, and JVM Overhead. On "
+ "containerized setups, this should be set to the container memory."
Comment on lines +260 to +263
Copy link
Contributor

@tillrohrmann tillrohrmann Jan 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would put the reference to flink.size at the end of the description similar to what we are doing in the description of TOTAL_FLINK_MEMORY.

);

/**
* Total Flink Memory size for the TaskExecutors.
*/
@Documentation.CommonOption(position = Documentation.CommonOption.POSITION_MEMORY)
public static final ConfigOption<MemorySize> TOTAL_FLINK_MEMORY =
key("taskmanager.memory.flink.size")
.memoryType()
.noDefaultValue()
.withDescription("Total Flink Memory size for the TaskExecutors. This includes all the memory that a"
+ " TaskExecutor consumes, except for JVM Metaspace and JVM Overhead. It consists of Framework Heap Memory,"
+ " Task Heap Memory, Task Off-Heap Memory, Managed Memory, and Network Memory.");
.memoryType()
.noDefaultValue()
.withDescription(String.format("Total Flink Memory size for the TaskExecutors. This includes all the "
+ "memory that a TaskExecutor consumes, except for JVM Metaspace and JVM Overhead. It consists of "
+ "Framework Heap Memory, Task Heap Memory, Task Off-Heap Memory, Managed Memory, and Network "
+ "Memory. See also '%s' for total process memory size configuration.",
TOTAL_PROCESS_MEMORY.key()));

/**
* Framework Heap Memory size for TaskExecutors.
Expand Down