-
Notifications
You must be signed in to change notification settings - Fork 12.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Consider specializing std::vec::from_elem to calloc when possible #38723
Comments
What's the benefit you are expecting? When would zeroed pages help in Vec? |
This would be targeting things like Calloc can avoid explicitly zeroing pages if it knows they're already zeroed out (e.g. if they just came from an mmap call). If much of the buffer isn't actually used, this can be way faster. See for example https://bitbucket.org/cffi/cffi/issues/295/cffinew-is-way-slower-than-it-should-be-it |
As a practical example, this could improve performance of both I asked about this before on one of the easy question threads on Reddit: one of the responses I got said that it probably wouldn't improve performance under jemalloc for various esoteric reasons (which I found hard to believe, but whatever); however, it would definitely help under the system allocator. Edit: consolidated two comments and removed redundant statements |
|
Specialize Vec::from_elem to use calloc Fixes rust-lang#38723. This specializes the implementation for `u8` only, but it could be extended to other zeroable types if desired. I haven't tested this extensively, but I did verify that it gives the expected performance boost for large `vec![0; n]` allocations with both alloc_system and jemalloc, on Linux. (I have not tested or even built the Windows code.)
Specialize Vec::from_elem to use calloc Fixes rust-lang#38723. This specializes the implementation for `u8` only, but it could be extended to other zeroable types if desired. I haven't tested this extensively, but I did verify that it gives the expected performance boost for large `vec![0; n]` allocations with both alloc_system and jemalloc, on Linux. (I have not tested or even built the Windows code.)
Specialize Vec::from_elem to use calloc Fixes rust-lang#38723. This specializes the implementation for `u8` only, but it could be extended to other zeroable types if desired. I haven't tested this extensively, but I did verify that it gives the expected performance boost for large `vec![0; n]` allocations with both alloc_system and jemalloc, on Linux. (I have not tested or even built the Windows code.)
Specialize Vec::from_elem to use calloc Fixes rust-lang#38723. This specializes the implementation for `u8` only, but it could be extended to other zeroable types if desired. I haven't tested this extensively, but I did verify that it gives the expected performance boost for large `vec![0; n]` allocations with both alloc_system and jemalloc, on Linux. (I have not tested or even built the Windows code.)
Specialize Vec::from_elem to use calloc Fixes rust-lang#38723. This specializes the implementation for `u8` only, but it could be extended to other zeroable types if desired. I haven't tested this extensively, but I did verify that it gives the expected performance boost for large `vec![0; n]` allocations with both alloc_system and jemalloc, on Linux. (I have not tested or even built the Windows code.)
Specialize Vec::from_elem to use calloc Fixes rust-lang#38723. This specializes the implementation for `u8` only, but it could be extended to other zeroable types if desired. I haven't tested this extensively, but I did verify that it gives the expected performance boost for large `vec![0; n]` allocations with both alloc_system and jemalloc, on Linux. (I have not tested or even built the Windows code.)
Specialize Vec::from_elem to use calloc Fixes #38723. This specializes the implementation for `u8` only, but it could be extended to other zeroable types if desired. I haven't tested this extensively, but I did verify that it gives the expected performance boost for large `vec![0; n]` allocations with both alloc_system and jemalloc, on Linux. (I have not tested or even built the Windows code.)
A solution for array where all is zero ? see the stackoverflow question: Efficiently allocating a vector of structs in Rust. Also, maybe add tuple. |
This will require adding calloc to the global allocator API, but can significantly speed up situations where a large buffer is allocated but not entirely used.
The text was updated successfully, but these errors were encountered: