Skip to content
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.

Reduce am resource #2072

Merged
merged 5 commits into from
Jan 31, 2019
Merged

Reduce am resource #2072

merged 5 commits into from
Jan 31, 2019

Conversation

mzmssg
Copy link
Member

@mzmssg mzmssg commented Jan 24, 2019

  1. reduce AM memory to 1G
  2. given Launcher AM might consume 3G+ virtual memory, increase YARN virtual memory ratio to 6, previous value is 2.1.

Test case:
A framework with 9 taskroles, 5 tasks per taskrole, 2-node cluster, for one hour.

@coveralls
Copy link

coveralls commented Jan 24, 2019

Coverage Status

Coverage increased (+0.2%) to 52.845% when pulling 97b76fd on zimiao/reduce_am_resource into f0fe0d9 on master.

@coveralls
Copy link

Coverage Status

Coverage remained the same at 52.68% when pulling 5d43fab on zimiao/reduce_am_resource into f0fe0d9 on master.

@Gerhut
Copy link
Member

Gerhut commented Jan 24, 2019

Shall we make it as default config in launcher?

@yqwang-ms
Copy link
Member

By setting this, it is already default for all jobs.

@mzmssg
Copy link
Member Author

mzmssg commented Jan 25, 2019

@Gerhut
Seems it would be better of putting it in job config instead hard-code here?

@mzmssg
Copy link
Member Author

mzmssg commented Jan 30, 2019

@Gerhut
Updated

@mzmssg mzmssg merged commit 1e6c84d into master Jan 31, 2019
@mzmssg mzmssg deleted the zimiao/reduce_am_resource branch March 22, 2019 08:22
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants