You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If self.kind() == KIND_VEC is true, freeze ends up calling vec.into() which appears to not be zero-cost at all, and ends up doing a huge memcpy somewhere in the bowels of Vec
By looking at it this comes from calling Vec::into_boxed_slice(), which calls Vec::shrink_to_fit().
That again means freeze is not zero cost if the BytesMut wasn't written up to it's capacity. If you reserve the exact amount upfront, it might not be an issue. If you split off anything upfront (which promotes to an Arc), then it also wouldn't be an issue.
This is all fixable. However a tricky question is whether people are all ok with unused memory in the BytesMut being "lost" while Bytes elements refer to a subset of it. For myself - I would be - given that I don't see Bytes as an ideal long term storage anyway and everyone should strive for releasing the chunks as soon as no longer required to avoid references keeping bigger buffers alive for longer than necessary.
After thinking a bit more about it: I think we can solve this easily by adding an additional shrink_and_freeze() method for people which care about potential waste of memory, and make sure freeze() is always allocation free.
Kixunil, koivunej, Jasper-Bekkers, djc, scottlamb and 4 more
After thinking a bit more about it: I think we can solve this easily by adding an additional shrink_and_freeze() method for people which care about potential waste of memory, and make sure freeze() is always allocation free.
Did this ever end up getting implemented, and if so, should we close this issue? ☺️
Activity
Jasper-Bekkers commentedon Jun 28, 2020
It looks like the original
from_vec
implementation didn't used to callinto_boxed_slice
before the refactoring in #298 https://github.com/tokio-rs/bytes/pull/298/files#diff-d272e47b868896a26bf5ddcfa9be0ea2L1836fanatid commentedon Jul 12, 2020
@Jasper-Bekkers out of interest, how you get this trace? Is there a way do nearly same on Linux?
Jasper-Bekkers commentedon Jul 12, 2020
It's profiled with Superluminal Profiler; https://superluminal.eu I don't think it supports Linux yet.
Matthias247 commentedon Oct 17, 2020
By looking at it this comes from calling
Vec::into_boxed_slice()
, which callsVec::shrink_to_fit()
.That again means
freeze
is not zero cost if theBytesMut
wasn't written up to it's capacity. If you reserve the exact amount upfront, it might not be an issue. If you split off anything upfront (which promotes to anArc
), then it also wouldn't be an issue.This is all fixable. However a tricky question is whether people are all ok with unused memory in the
BytesMut
being "lost" whileBytes
elements refer to a subset of it. For myself - I would be - given that I don't seeBytes
as an ideal long term storage anyway and everyone should strive for releasing the chunks as soon as no longer required to avoid references keeping bigger buffers alive for longer than necessary.Matthias247 commentedon Oct 17, 2020
After thinking a bit more about it: I think we can solve this easily by adding an additional
shrink_and_freeze()
method for people which care about potential waste of memory, and make surefreeze()
is always allocation free.Jasper-Bekkers commentedon Jan 25, 2022
Did this ever end up getting implemented, and if so, should we close this issue?☺️