[FLINK-30328][tests] Fix unstable TaskManagerWideRocksDbMemorySharingITCase #21778
Conversation
|
@AlanConfluent could you please review this PR? |
rkhachatryan
force-pushed
the
f30328-extract-interface
branch
2 times, most recently
from
adba5d4
to
d9b1c9e
Compare
| } | ||
| Assert.assertEquals(1, createdCaches.size()); |
There was a problem hiding this comment.
Nice, this seems like a much more reliable way to check the cache directly.
There was a problem hiding this comment.
Presumably, if they were not sharing the cache as intended, you would have more than one in this list right?
There was a problem hiding this comment.
Right: there will be a cache per task if memory is shared inside slot, and no caches if memory is not shared.
...sdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBMemoryControllerUtils.java
Outdated
Show resolved
Hide resolved
...sdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBMemoryControllerUtils.java
Show resolved
Hide resolved
...ksdb/src/main/java/org/apache/flink/contrib/streaming/state/EmbeddedRocksDBStateBackend.java
Outdated
Show resolved
Hide resolved
...ksdb/src/main/java/org/apache/flink/contrib/streaming/state/EmbeddedRocksDBStateBackend.java
Outdated
Show resolved
Hide resolved
rkhachatryan
force-pushed
the
f30328-extract-interface
branch
from
e15f8db
to
8a94b10
Compare
|
RocksDB actually cannot limit the native memory usage perfectly, why we can make the test stable if not leveraging metrics to detect the memory usage? |
This PR changes the test from measuring memory usage to directly verifying the number of Cache/WBM objects that were actually created (regardless of capacity). Sharing of those objects was the goal of the main PR. |
|
CI failures are caused by FLINK-29427 - waiting to be resolved |
… to facilitate testing
…erWideRocksDbMemorySharingITCase
rkhachatryan
force-pushed
the
f30328-extract-interface
branch
from
8a94b10
to
69c0460
Compare
TaskManagerWideRocksDbMemorySharingITCaseis unstable because it uses metrics to validate RocksDB cache/wbm sharing.This PR extract an interface to create those shared objects (WBM/cache) and configures state backend to use its test implementation.
As a side-effect, mocking is removed from
RocksDBMemoryControllerUtilsTest.cc: @Myasuka