Skip to content

Commit

Permalink
fixup! [FLINK-15143][docs] Add migration guide from pre-FLIP-49 memor…
Browse files Browse the repository at this point in the history
…y config
  • Loading branch information
azagrebin committed Feb 7, 2020
1 parent 16eeb75 commit e34dafa
Showing 1 changed file with 11 additions and 11 deletions.
22 changes: 11 additions & 11 deletions docs/ops/memory/mem_migration.md
Expand Up @@ -41,7 +41,7 @@ that at least one subset of the following options is configured explicitly, othe
* [`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)
* [`taskmanager.memory.task.heap.size`](../config.html#taskmanager-memory-task-heap-size) and [`taskmanager.memory.managed.size`](../config.html#taskmanager-memory-managed-size)

The [default flink-conf.yaml](#default-configuration-in-flink-confyaml) shipped with Flink sets [`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)
The [default `flink-conf.yaml`](#default-configuration-in-flink-confyaml) shipped with Flink sets [`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)
to make the default memory configuration consistent.

This [spreadsheet](https://docs.google.com/spreadsheets/d/1mJaMkMPfDJJ-w6nMXALYmTc4XxiV30P5U7DzgwLkSoE) can also help
Expand Down Expand Up @@ -130,10 +130,10 @@ but they still have the same semantics for the job manager process. See also [ho

## Total Memory (previously Heap Memory)

The previous options which were responsible for the total memory used by Flink are taskmanager.heap.size or taskmanager.heap.mb.
The previous options which were responsible for the total memory used by Flink are `taskmanager.heap.size` or `taskmanager.heap.mb`.
Despite their naming, they included not only JVM heap but also other off-heap memory components. The options have been deprecated.

The Mesos integration also had a separate option with the same semantics: mesos.resourcemanager.tasks.mem which has also been deprecated.
The Mesos integration also had a separate option with the same semantics: `mesos.resourcemanager.tasks.mem` which has also been deprecated.

If the mentioned legacy options are used without specifying the corresponding new options,
they will be directly translated into the following new options:
Expand Down Expand Up @@ -169,13 +169,13 @@ See also [how to configure managed memory now](mem_setup.html#managed-memory).

### Explicit size

The previous option to configure managed memory size (taskmanager.memory.size) was renamed to
The previous option to configure managed memory size (`taskmanager.memory.size`) was renamed to
[`taskmanager.memory.managed.size`](../config.html#taskmanager-memory-managed-size) and deprecated.
It is recommended to use the new option because the legacy one can be removed in future releases.

### Fraction

If not set explicitly, the managed memory could be previously specified as a fraction (taskmanager.memory.fraction)
If not set explicitly, the managed memory could be previously specified as a fraction (`taskmanager.memory.fraction`)
of the total memory minus network memory and container cut-off (only for [Yarn](../deployment/yarn_setup.html) and
[Mesos](../deployment/mesos.html) deployments). This option has been completely removed and will have no effect if still used.
Please, use the new option [`taskmanager.memory.managed.fraction`](../config.html#taskmanager-memory-managed-fraction) instead.
Expand All @@ -194,9 +194,9 @@ See also [how to migrate container cut-off](#container-cut-off-memory).
### Other changes

Additionally, the following changes have been made:
* The [managed memory](mem_setup.html#managed-memory) is always off-heap now. The configuration option taskmanager.memory.off-heap is removed and will have no effect anymore.
* The [managed memory](mem_setup.html#managed-memory) is always off-heap now. The configuration option `taskmanager.memory.off-heap` is removed and will have no effect anymore.
* The [managed memory](mem_setup.html#managed-memory) now uses native memory which is not direct memory. It means that the managed memory is no longer accounted for in the JVM direct memory limit.
* The [managed memory](mem_setup.html#managed-memory) is always lazily allocated now. The configuration option taskmanager.memory.preallocate is removed and will have no effect anymore.
* The [managed memory](mem_setup.html#managed-memory) is always lazily allocated now. The configuration option `taskmanager.memory.preallocate` is removed and will have no effect anymore.

## Container Cut-Off Memory

Expand All @@ -222,13 +222,13 @@ The mentioned configuration options remain valid for the job manager in the same

## Default Configuration in flink-conf.yaml

This section describes the changes of the default flink-conf.yaml shipped with Flink.
This section describes the changes of the default `flink-conf.yaml` shipped with Flink.

The total memory (taskmanager.heap.size) is replaced by [`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)
in the default flink-conf.yaml. The value is also increased from 1024Mb to 1568Mb.
The total memory (`taskmanager.heap.size`) is replaced by [`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)
in the default `flink-conf.yaml`. The value is also increased from 1024Mb to 1568Mb.
See also [how to configure total memory now](mem_setup.html#configure-total-memory).

<div class="alert alert-warning">
<strong>Warning:</strong> If you use the new default flink-conf.yaml it can result in different sizes of
<strong>Warning:</strong> If you use the new default `flink-conf.yaml` it can result in different sizes of
the memory components and can lead to performance changes.
</div>

0 comments on commit e34dafa

Please sign in to comment.