Add API to configure memory allocator settings #453
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
torch.mps.memory.set_allocator_settings(str)
to adjust high/low watermark ratios, small/large/xlarge heap size divisors, max pow2 roundup size, and debug verbosity of MPSAllocator messages.PYTORCH_MPS_ALLOC_CONF=[settings string]
to pass the settings string as an alternative to the APIset_allocator_settings()
(similar to CUDA).PYTORCH_DEBUG_MPS_ALLOCATOR
,PYTORCH_MPS_HIGH_WATERMARK_RATIO
, andPYTORCH_MPS_LOW_WATERMARK_RATIO
.total_memory_allocated_size
to count the total size of MTLHeaps and not MTLBuffers (in case heaps aren't fully occupied).