Smaller memory footprint for BoundedArray - #16299
Conversation
Co-authored-by: zooster <r00ster91@proton.me>
andrewrk
left a comment
There was a problem hiding this comment.
All these += @intCast(b) are actually 2 safety checks. This can produce better code if you do this instead:
x = @intCast(x + b);
|
Thank you! This is a nice enhancement. |
|
@andrewrk this tripped a pretty bad bug for us in TigerBeetle: Essentially, we were doing I don't think this is a problem with this change per se, as it all is very much reasonable. But it does seem that integer overflow semantics might need some extra work here. |
|
Although I am wondering if it perhaps makes sense to do len_repr: Len
pub inline fn len(self: BoundedArray) usize {
return self.len_repr
}in the meantime, so that the call-sites are more aware that something funky with integrer ranges might be happening here? |
Changing the representation of `len` made it unsafe to add slice lengths. That silently introduced integer overflows in `hpke`. Change was introduced in ziglang/zig#16299
This reverts commit cb5a6be. I deeply apologize for the churn. This change is problematic given that we do not have ranged integers (yet? see #3806). In the meantime, this type needs to be `usize`, matching the length and index types for all std lib data structures. Users who want to save memory should not use heap-allocated BoundedArray values, since it is inherently memory-inefficient. Use a different memory layout instead. If #3806 is accepted and implemented, the length value can become an integer with the appropriate range, without the footgun. If that proposal is not accepted, len type will remain a usize.
|
Reverted in 85747b2. |
This PR changes
BoundedArray'slenfield from ausizeto a minimum-sized integer. If you're keeping a million of these in memory, their size can add up.BoundedArray(u8, 3)used to take 16 bytes, but now (withlenstored as au2) it only takes 4 bytes.Notes:
lenis the only type that changed. I left all theusizeparameters alone, to not break existing code. A new test was added, and the old tests were not changed.@intCastis sprinkled in. For each one, either the value had already checked before that point, or it's in an "assume" method.appendNTimesAssumeCapacityis an "assume" method, but it had an assert inside. Am I right to think it's odd to mix assumes with asserts? Either way I left the assert as it was.This PR message is now longer than the diff, oops. I'll stop rambling. Happy reviewing!