[HIVE-21686] ensure that memory allocator does not evict using brute …#626
[HIVE-21686] ensure that memory allocator does not evict using brute …#626b-slim wants to merge 6 commits intoapache:masterfrom
Conversation
…foce path. Change-Id: I020d9be9e76ac095f909b53cbec9e63f21efe8ff
odraese
left a comment
odraese
left a comment
There was a problem hiding this comment.
Mainly minor stuff, except the one portion of the
allocLog2 << dest.length - destAllocIx + forceEvictAttempt - MAX_FAST_ATTEMPT
which I either don't understand or which is wrong....
llap-server/src/java/org/apache/hadoop/hive/llap/cache/BuddyAllocator.java
Outdated
Show resolved
Hide resolved
llap-server/src/java/org/apache/hadoop/hive/llap/cache/BuddyAllocator.java
Outdated
Show resolved
Hide resolved
llap-server/src/java/org/apache/hadoop/hive/llap/cache/BuddyAllocator.java
Outdated
Show resolved
Hide resolved
llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelCacheMemoryManager.java
Outdated
Show resolved
Hide resolved
| * @param memoryToReserve amount of bytes to evict | ||
| * @return actual amount of evicted bytes. | ||
| */ | ||
| long evictMemory(long memoryToReserve); |
There was a problem hiding this comment.
Adding this to MemoryManager kind'o breaks the separation of memory manager and eviction policy (at least from interface perspective, these were two separate things so far).
There was a problem hiding this comment.
Memory manager is doing eviction on reserve call, thus this is adding an explicit handle to the eviction without reserving it. do you a different idea?
llap-server/src/test/org/apache/hadoop/hive/llap/cache/TestBuddyAllocator.java
Outdated
Show resolved
Hide resolved
llap-server/src/test/org/apache/hadoop/hive/llap/cache/TestCacheAllocationsEvictionsCycles.java
Outdated
Show resolved
Hide resolved
llap-server/src/test/org/apache/hadoop/hive/llap/cache/TestOrcMetadataCache.java
Outdated
Show resolved
Hide resolved
Change-Id: I5664256e4350536546cdee9a92fb7a39df2c9980
Change-Id: I1ffc034b98f5f008d0b54fd1b6006dbbb7b5b4ad
Change-Id: Ibeacf2a29cc6bde452a64b548c4fa49d4a4ce870
Change-Id: I7ab1279a3375c3cf2f36bf75dc798cb35bb91a9f
Change-Id: I7a14811c2c04e95f82ccd58b162ea38b398691d9
|
@odraese Thanks for the review, i did fix most of your comments can you please take a second look when you have time. |
odraese
left a comment
odraese
left a comment
There was a problem hiding this comment.
Looking good. Thx for the changes!
|
thanks pushed to master |
2 participants
[HIVE-21686] ensure that memory allocator does not evict using brute foce path.
Change-Id: I020d9be9e76ac095f909b53cbec9e63f21efe8ff