[FLINK-20663][runtime] Instantly release unsafe memory on freeing segment. #14904

xintongsong · 2021-02-09T03:35:43Z

Brief change log

Do not allow wrapping buffers from unsafe segments.
Release unsafe memory instantly on segments freed.

flinkbot · 2021-02-09T03:38:19Z

Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.

Automated Checks

Last check on commit e4771ac (Fri Feb 19 07:33:15 UTC 2021)

Warnings:

No documentation files were touched! Remember to keep the Flink docs up to date!

_{Mention the bot in a comment to re-run the automated checks.}

Review Progress

❓ 1. The [description] looks good.
❓ 2. There is [consensus] that the contribution should go into to Flink.
❓ 3. Needs [attention] from.
❓ 4. The change fits into the overall [architecture].
❓ 5. Overall code [quality] is good.

Please see the Pull Request Review Guide for a full explanation of the review process.

The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot approve description to approve one or more aspects (aspects: description, consensus, architecture and quality)
@flinkbot approve all to approve all aspects
@flinkbot approve-until architecture to approve everything until architecture
@flinkbot attention @username1 [@username2 ..] to require somebody's attention
@flinkbot disapprove architecture to remove an approval you gave earlier

flinkbot · 2021-02-09T03:47:01Z

CI report:

d1936a8 Azure: SUCCESS
e4771ac Azure: PENDING

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot run travis re-run the last Travis build
@flinkbot run azure re-run the last Azure build

xintongsong · 2021-02-09T08:55:57Z

cc @tillrohrmann
Could you please take a look at this PR?

One thing I'm not entirely sure is that, do we still need a configuration option to fallback to the original behaviors, given that we have already forbidden wrapping buffers from unsafe segments?

This branch contains an additional commit compared to this PR, which introduces such a configuration option by touching quite some files. (FYI, this branch also passed the CI tests.)

tillrohrmann

Thanks for creating this PR @xintongsong. Assuming that nobody calls wrap on an unsafe memory segment, I think the solution should work. I think the solution with processAsByteBuffer is actually quite nice :-)

I had a couple of comments concerning the remaining wrap calls and the costs of the processAsByteBuffer method. Please take a look.

One additional thought is whether we couldn't solve the problem a bit differently by changing how UnsafeMemoryBudget works. If we had two counters expectedFreeMemory and actuallyAvailableMemory where expectedFreeMemory is the memory which should be available but has not necessarily been freed yet, we could wait indefinitely in the UnsafeMemoryBudget.reserveMemory method until the memory has been returned.

Whenever we call MemorySegment.free we increase the UnsafeMemoryBudget.expectedFreeMemory. Whenever the cleaner of the unsafe memory runs, then we increase actuallyAvailableMemory. That way we should always know when to expect more memory to be freed by the GC procedure and, thus, can decide when to wait and when not to wait for the memory.

I am not saying that we have to do it like this but I wanted to make sure that we had this solution on the radar.

flink-core/src/main/java/org/apache/flink/core/memory/MemorySegment.java

tillrohrmann · 2021-02-12T15:13:12Z

flink-core/src/main/java/org/apache/flink/core/memory/HybridMemorySegment.java

+    @Override
+    public <T> T processAsByteBuffer(Function<ByteBuffer, T> processFunction) {
+        return Preconditions.checkNotNull(processFunction).apply(wrapInternal(0, size));
+    }


Which calls to wrap cannot be replaced with this call?

Do we know what the performance penalty for the additional Function is?

Which calls to wrap cannot be replaced with this call?

Technically, I don't think there are wrap calls that CANNOT be replaced. Some of the calls only wrap part of the segment, which can be solved by simply adding offset and length as arguments. I have not replaced all calls to wrap with this processAsByteBuffer, mostly because the remaining calls are on direct segment, thus it's not necessary to migrate them.

Do we know what the performance penalty for the additional Function is?

We don't have a measurement. However, given that this method is for the per-segment operation (by default 32KB), I don't expect a significant performance penalty from using the Function.

We could think about replacing these calls with processAsByteBuffer also for the direct memory in order to be able to get rid of the wrap method.

I'll look into them together with the follow-up refactor effort. Since not all wrap calls are on the entire segment, I'll need to check the performance impact from using Function on those calls.

Maybe it is also ok to provide additional methods on the MemorySegment which use internally a ByteBuffer implementation to be efficient.

tillrohrmann · 2021-02-12T15:15:56Z

flink-core/src/main/java/org/apache/flink/core/memory/HybridMemorySegment.java

+        if (cleaner != null) {
+            cleaner.run();
+        }


Looking at the special casing of this class, why don't we introduce an DirectMemorySegment and a UnsafeMemorySegment? Also, why do we have the HeapMemorySegment and still allow the HybridMemorySegment to be used with heap memory?

If we had a DirectMemorySegment, then we could only allow wrap on this type, for example.

HybridMemorySegment is currently the only MemorySegment implementation that is used in production. HeapMemorySegment is no longer used, but seems intentionally not removed as it is annotated with @SuppressWarnings("unused").

From my side, it makes sense to split HybridMemorySegment into HeapMemorySegment, DirectMemorySegment and UnsafeMemorySegment. This should get rid of lots of if-else branches in the implementation.

I'm leaning towards addressing this refactor as a follow-up effort, and only do that for the master branch.

Ok, this sounds reasonable. Then let's do this refactoring soon. In fact, HeapMemorySegment does not seem to be that unused anymore. Somehow the table API started using this class for some RawFormatDeserializationSchema. This might be a good indicator for pulling things straight.

With this refactoring we could also think about getting rid of the whole native memory cleaner logic.

I'll drive the refactor effort and try to get it done in the 1.13 release cycle.

flink-core/src/main/java/org/apache/flink/core/memory/HybridMemorySegment.java

xintongsong · 2021-02-14T03:41:37Z

@tillrohrmann Thanks for the review.

Concerning indefinitely waiting when there's enough expectedFreeMemory but not enough actualFreeMemory, we have actually considered similar approach.

I think this approach indeed solves the failures due to GC not releasing memory timely. However, it solves the problem by waiting longer, which may not have as good performance as the current solution that tries to release earlier. In case there's no enough heap activities to trigger GC in advance, we may frequently experience full GC on finishing existing tasks and deploying new tasks, which is costly for batch scenarios where deploying new tasks are frequent and TMs are usually large.

xintongsong · 2021-02-14T04:08:55Z

Comments addressed.

tillrohrmann

Thanks for the further explanation. I am ok with the proposed changes. Maybe we could add one additional test which ensures that the cleaner is called when calling free.

I would also be in favour of doing the refactoring for the HybridMemorySegment soonish because it feels that this class is used a bit differently now compared to when it was written.

flink-core/src/test/java/org/apache/flink/core/memory/HybridOffHeapUnsafeMemorySegmentTest.java

This method is only used in tests.

…orySegment#processAsByteBuffer.

… segments.

… freed.

xintongsong · 2021-02-15T10:34:48Z

Thanks for the review, @tillrohrmann.
I've added the test case. Will merge this once AZP gives green light.
For the follow-up refactor effort, I opened FLINK-21375.

xintongsong · 2021-02-15T15:21:12Z

Passed CI here:

master (1.13): https://dev.azure.com/tonysong820/Flink/_build/results?buildId=157&view=results
release-1.12: https://dev.azure.com/tonysong820/Flink/_build/results?buildId=158&view=results

Merging this.

This closes #14904

curcur · 2021-02-15T15:46:22Z

thanks!

rmetzger added the review=description? label Feb 9, 2021

rmetzger added the component=Runtime/Task label Feb 9, 2021

xintongsong force-pushed the FLINK-20663-managed-memory-1 branch 2 times, most recently from b194119 to ce6b3c3 Compare February 9, 2021 08:46

xintongsong changed the title ~~[WIP][FLINK-20663][runtime] Instantly release unsafe memory on freeing segment.~~ [FLINK-20663][runtime] Instantly release unsafe memory on freeing segment. Feb 9, 2021

xintongsong marked this pull request as ready for review February 9, 2021 08:49

tillrohrmann self-assigned this Feb 12, 2021

tillrohrmann reviewed Feb 12, 2021

View reviewed changes

xintongsong force-pushed the FLINK-20663-managed-memory-1 branch from ce6b3c3 to d1936a8 Compare February 14, 2021 04:08

tillrohrmann approved these changes Feb 15, 2021

View reviewed changes

flink-core/src/test/java/org/apache/flink/core/memory/HybridOffHeapUnsafeMemorySegmentTest.java Show resolved Hide resolved

xintongsong added 6 commits February 15, 2021 18:21

[hotfix][core] Remove unused HybridMemorySegment#getOffHeapBuffer.

1516483

This method is only used in tests.

[FLINK-20663][core] Introduce MemorySegment#processAsByteBuffer.

5e76cda

[FLINK-20663][runtime] Migrate use cases of MemorySegment#wrap to Mem…

c7776ff

…orySegment#processAsByteBuffer.

[FLINK-20663][core] Forbid calling HybridMemorySegment#wrap on unsafe…

2737a82

… segments.

[FLINK-20663][core] Release unsafe memory instantly on segment freed.

ce98105

fixup! [FLINK-20663][core] Release unsafe memory instantly on segment…

e4771ac

… freed.

xintongsong force-pushed the FLINK-20663-managed-memory-1 branch from d1936a8 to e4771ac Compare February 15, 2021 10:32

xintongsong closed this in 75fc592 Feb 15, 2021

xintongsong added a commit that referenced this pull request Feb 15, 2021

[FLINK-20663][core] Release unsafe memory instantly on segment freed.

1f8be1f

This closes #14904

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FLINK-20663][runtime] Instantly release unsafe memory on freeing segment. #14904

[FLINK-20663][runtime] Instantly release unsafe memory on freeing segment. #14904

xintongsong commented Feb 9, 2021 •

edited

flinkbot commented Feb 9, 2021 •

edited

flinkbot commented Feb 9, 2021 •

edited

xintongsong commented Feb 9, 2021 •

edited

tillrohrmann left a comment •

edited

tillrohrmann Feb 12, 2021

tillrohrmann Feb 12, 2021

xintongsong Feb 14, 2021

tillrohrmann Feb 15, 2021

xintongsong Feb 15, 2021

tillrohrmann Feb 15, 2021

tillrohrmann Feb 12, 2021

tillrohrmann Feb 12, 2021

xintongsong Feb 14, 2021

tillrohrmann Feb 15, 2021

tillrohrmann Feb 15, 2021

xintongsong Feb 15, 2021

xintongsong commented Feb 14, 2021

xintongsong commented Feb 14, 2021

tillrohrmann left a comment

xintongsong commented Feb 15, 2021

xintongsong commented Feb 15, 2021 •

edited

curcur commented Feb 15, 2021

[FLINK-20663][runtime] Instantly release unsafe memory on freeing segment. #14904

[FLINK-20663][runtime] Instantly release unsafe memory on freeing segment. #14904

Conversation

xintongsong commented Feb 9, 2021 • edited

Brief change log

flinkbot commented Feb 9, 2021 • edited

Automated Checks

Review Progress

flinkbot commented Feb 9, 2021 • edited

CI report:

xintongsong commented Feb 9, 2021 • edited

tillrohrmann left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xintongsong commented Feb 14, 2021

xintongsong commented Feb 14, 2021

tillrohrmann left a comment

Choose a reason for hiding this comment

xintongsong commented Feb 15, 2021

xintongsong commented Feb 15, 2021 • edited

curcur commented Feb 15, 2021

xintongsong commented Feb 9, 2021 •

edited

flinkbot commented Feb 9, 2021 •

edited

flinkbot commented Feb 9, 2021 •

edited

xintongsong commented Feb 9, 2021 •

edited

tillrohrmann left a comment •

edited

xintongsong commented Feb 15, 2021 •

edited