You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have searched in the issues and found no similar issues.
What would you like to be improved?
When the shuffle server is deployed in the container, the memory keeps soaring until it exceeds the container's memory limit, and then the container is OOMKill. After checking the memory usage, it is found that there are a large number of 64 MB memory blocks.
Reference: prestodb/presto#8993 https://issues.apache.org/jira/browse/HADOOP-7154
How should we improve?
In start-shuffle-server.sh, set MALLOC_ARENA_MAX(default cpu core * 8) to control memory application/usage
Are you willing to submit PR?
Yes I am willing to submit a PR!
The text was updated successfully, but these errors were encountered:
…860)
### What changes were proposed in this pull request?
Reduce memory usage for some versions of glibc. Prevent oom.
### Why are the changes needed?
Fix: #859
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
UT
Co-authored-by: haorenhui <haorenhui@kingsoft.com>
Code of Conduct
Search before asking
What would you like to be improved?
When the shuffle server is deployed in the container, the memory keeps soaring until it exceeds the container's memory limit, and then the container is OOMKill. After checking the memory usage, it is found that there are a large number of 64 MB memory blocks.
![image](https://private-user-images.githubusercontent.com/41908930/237272543-fd9aebeb-3417-483d-b55a-4d98ad38d81d.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjE1NTI5NDIsIm5iZiI6MTcyMTU1MjY0MiwicGF0aCI6Ii80MTkwODkzMC8yMzcyNzI1NDMtZmQ5YWViZWItMzQxNy00ODNkLWI1NWEtNGQ5OGFkMzhkODFkLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA3MjElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNzIxVDA5MDQwMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTJiYzY5NzMwYjJhYWRhYzJiZjAyMTFkNDNkN2U5ZDkyZDJlODA3NDQ1Njk5ODYxYWQ2Njg3YmY0NTc1ZjMzYmMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.64geLI8Gd0ZMNKWsjc_Mae2e0cbBDs53M57cjmGwa2o)
Reference:
prestodb/presto#8993
https://issues.apache.org/jira/browse/HADOOP-7154
How should we improve?
In start-shuffle-server.sh, set MALLOC_ARENA_MAX(default cpu core * 8) to control memory application/usage
Are you willing to submit PR?
The text was updated successfully, but these errors were encountered: