-
Notifications
You must be signed in to change notification settings - Fork 198
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Enable pool_memory_resource
to deallocate to upstream
#571
Conversation
Please update the changelog in order to start CI tests. View the gpuCI docs here. |
Testing |
This PR has been labeled |
The optimization in this PR was moved to #702 . Testing of this PR found that freeing upstream leads to poor performance due to thrashing. We decided that instead we should look into making an explicit way to request the pool is compacted and trimmed. This feature is on hold. Closing. |
Previously, once the pool memory resource needed to grow its pool from upstream, it could not shrink the pool until the MR was destroyed.
This is a WIP to enable freeing upstream blocks back to the upstream resource, which should enable more flexible composition of resources. Initial benchmarking shows that performance is no worse in RANDOM_ALLOCATIONS_BENCH with this change, and possibly even slightly better.
Next, I realized that the
pool_memory_resource
maintains a set of allocated blocks that is used for nothing other than being able to print out allocated blocks for debugging. So I moved this behind a defineTRACK_ALLOCATIONS
which is undefined by default. This is a significant performance benefit (up to 36% faster on RANDOM_ALLOCATIONS_BENCH). Full comparison (second column is % difference in time between new and old -- negative values are improvements).TODO