Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Give the user the ability to allocate all memory himself #81

Open
cospectrum opened this issue Sep 22, 2023 · 4 comments
Open

Give the user the ability to allocate all memory himself #81

cospectrum opened this issue Sep 22, 2023 · 4 comments

Comments

@cospectrum
Copy link

Hello! I noticed that each gemm call allocates memory on the heap for "packing buffers".
So maybe there is a way to give the user the ability to pre-allocate all the necessary memory and simply pass it as an argument without using a global allocator?

For example, add helper function(s) to the public api to calculate the required memory for a particular type/shape/kernel, and then initialize aligned_alloc::Alloc struct from a pointer or something like that.

I think this feature can help use the crate in real-time programs where hidden allocations are not welcome

@cospectrum cospectrum changed the title Give the user the opportunity to allocate all memory himself Give the user the ability to allocate all memory himself Sep 22, 2023
@sainteckes
Copy link

I would like to use it in realtime as well, so +1 on that :)

@cospectrum
Copy link
Author

If anyone is interested, I created microgemm.
It does not make allocations on its own.
I'm trying to keep it very flexible and avoid unsafe

@cospectrum
Copy link
Author

cospectrum commented Mar 5, 2024

Custom allocators would help. Maybe allocator-api2

@sainteckes
Copy link

@cospectrum On microcontrollers I was using matrixmultiply already with this allocator: Buddy System Allocator.
Do you think I can just use this allocator on desktop and then it is "real-time" safe, because it is taking the memory from the pre-allocated memory?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants