-
Notifications
You must be signed in to change notification settings - Fork 498
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add flags to set caching memory manager split+recycling size limits #420
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@avidov has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
a95731a
to
ca11228
Compare
@avidov has updated the pull request. You must reimport the pull request before landing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@avidov has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
ca11228
to
34bcd14
Compare
@avidov has updated the pull request. You must reimport the pull request before landing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@avidov has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
Add flags to set configurable limits of the CachingMemoryManager. These flags allow customization for different workloads. This diff changes the default recycling size limit for training to 256MB. Following investigation of OOM on dynamic batching nd based on past allocation analyses we found that this limit works well. Big thanks for WilliamTambellini@ for his work on improving the memory manager capability in handling flexible workload: flashlight#188
34bcd14
to
cb0d3c7
Compare
@avidov has updated the pull request. You must reimport the pull request before landing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@avidov has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
Hi @avidov! Thank you for your pull request. We require contributors to sign our Contributor License Agreement, and yours needs attention. You currently have a record in our system, but the CLA is no longer valid, and will need to be resubmitted. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
Add flags to set configurable limits of the CachingMemoryManager. These flags
allow customization for different workloads. This diff changes the default recycling
size limit for training to 256MB. Following investigation of OOM on dynamic batching
nd based on past allocation analyses we found that this limit works well.
Big thanks for WilliamTambellini@ for his work on improving the memory manager
capability in handling flexible workload:
#188
IMPORTANT: Please do not create a Pull Request without creating an issue first. Changes must be discussed.
Original Issue: [corresponding issue on Github]
Note: You can add
closes #[issue number]
to automatically close the issue that this PR resolves when it is merged.Summary
[Explain the details for this change and the problem that the pull request solves]
Test Plan (required)
[steps by which you tested that your fix resolves the issue. These might include specific commands and configurations]