Skip to content

[MPS] Improve memory usage and performance utilizing garbage collector and adaptive commit #86119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

[MPS] Improve memory usage and performance utilizing garbage collector and adaptive commit #86119

wants to merge 1 commit into from

Conversation

razarmehr
Copy link
Collaborator

  • Improve memory usage and performance utilizing garbage collector and adaptive commit
  • Enable low watermark limit to detect memory pressure.
  • Enable garbage collection and adaptive commit strategies when under memory pressure.
  • More efficient resource management by splitting large heaps (instead of reusing oversized buffers for smaller allocation requests)
  • Introduce Extra Large heaps to improve performance by avoiding numerous costly allocation of smaller heaps
  • Fix purgeability when releasing the Metal heaps
  • Fix the race condition when deferring the heap's size update

Fixes #79283

@razarmehr
Improve memory usage and performance utilizing garbage collector and …
1fa797c 
…adaptive commit (#113)

* Improve memory usage and performance utilizing garbage collector and adaptive commit
- Enable low watermark limit to detect memory pressure.
- Enable garbage collection and adaptive commit strategies when under memory pressure.
- More efficient resource management by splitting large heaps (instead of reusing oversized buffers for smaller allocation requests)
- Introduce Extra Large heaps to improve performance by avoiding numerous costly allocation of smaller heaps
- Fix purgeability when releasing the Metal heaps

* Fix the race condition when deferring the heap's size update
@razarmehr razarmehr added the ciflow/mps Run MPS tests (subset of trunk) label Oct 3, 2022
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 3, 2022 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/86119

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

❌ 2 Failures

As of commit 1fa797c:

The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added the release notes: mps Release notes category label Oct 3, 2022
kulinseth
kulinseth approved these changes Oct 3, 2022
View reviewed changes
Copy link
Collaborator

@kulinseth kulinseth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine.

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Oct 3, 2022
@kulinseth kulinseth changed the title Improve memory usage and performance utilizing garbage collector and adaptive commit [MPS] Improve memory usage and performance utilizing garbage collector and adaptive commit Oct 3, 2022
@malfet
Copy link
Contributor

malfet commented Oct 3, 2022

@pytorchbot merge -f "MPS tests are green"

@pytorchmergebot
Copy link
Collaborator

@pytorchbot successfully started a merge job. Check the current status here.
The merge job was triggered with the force (-f) flag. This means your change will be merged immediately, bypassing any CI checks (ETA: 1-5 minutes). If this is not the intended behavior, feel free to use some of the other merge options in the wiki.
Please reach out to the PyTorch DevX Team with feedback or questions!

@github-actions
Copy link
Contributor

github-actions bot commented Oct 3, 2022

Hey @razarmehr.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

@razarmehr razarmehr deleted the mps_garbage_collector branch October 3, 2022 19:31
@dfalbel
Copy link
Contributor

dfalbel commented Oct 5, 2022 •
edited
Loading

@razarmehr Do you think we could still call the memory callbacks registered here:

pytorch/aten/src/ATen/mps/MPSAllocator.mm

Line 12 in fc94a21

C10_DEFINE_REGISTRY(MPSAllocatorCallbacksRegistry, IMpsAllocatorCallback);

Looks like their usage have been removed in this PR.

@dfalbel dfalbel mentioned this pull request Oct 6, 2022
Merged
@dfalbel dfalbel mentioned this pull request Oct 20, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kulinseth kulinseth kulinseth approved these changes

@malfet malfet malfet approved these changes

@DenisVieriu97 DenisVieriu97 Awaiting requested review from DenisVieriu97

Assignees
No one assigned
Labels
ciflow/mps Run MPS tests (subset of trunk) ciflow/trunk Trigger trunk jobs on your pull request cla signed Merged open source release notes: mps Release notes category
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

OOM Issues despite plenty of resources on M1 Mac
7 participants
@razarmehr @malfet @pytorchmergebot @dfalbel @kulinseth @facebook-github-bot @pytorchbot