Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make @memcpy and @memset use slices with any element type instead of pointers with bytes #14040

Closed
andrewrk opened this issue Dec 23, 2022 · 2 comments · Fixed by #15278
Closed
Labels
accepted This proposal is planned. breaking Implementing this issue could cause existing code to no longer compile or have different behavior. enhancement Solving this issue will likely involve adding new logic or components to the codebase. proposal This issue suggests modifications. If it also has the "accepted" label then it is planned.
Milestone

Comments

@andrewrk
Copy link
Member

New prototypes:

@memcpy(dest, src)

dest must be a slice of any element type.

src must be a pointer or a slice. It gets sliced from 0 to the same length as dest, triggering the same set of safety checks and possible compile errors as src[0..dest.len] would.

It is illegal for dest and src[0..dest.len] to overlap. If safety checks are enabled, there will be a runtime check for such overlapping.

@memset(dest, value)

dest must be a slice of any element type.

value is coerced to the dest element type.

Motivation

Such operations are common, and currently std.mem.copy and std.mem.set are used for this purpose. However, std.mem.copy does not have the non-overlapping requirement. This restriction allows for better code generation and additionally allows lowerings to call directly into aligned memset/memcpy implementations without runtime branching.

For example, @memcpy used to copy objects of type struct { x: u64, y: u64 }, when targeting a CPU with avx512 instructions, could lower directly to a memcpy implementation which uses these instructions and would not need any extra logic to handle the beginning or end of the memcpy operation.

In summary, this would both increase convenience for the programmer as well as the compiler's ability to optimize.

Implementation Notes

Be sure to set the alignment of the memcpy to the minimum alignment of the two slices, taking into account over-aligned pointers. LLVM's memcpy intrinsic does not have the ability to annotate that the number of bytes will be a multiple of the element size, so this is potentially an advantage Zig codegen backends could have over LLVM.

@andrewrk andrewrk added enhancement Solving this issue will likely involve adding new logic or components to the codebase. breaking Implementing this issue could cause existing code to no longer compile or have different behavior. proposal This issue suggests modifications. If it also has the "accepted" label then it is planned. accepted This proposal is planned. labels Dec 23, 2022
@andrewrk andrewrk added this to the 0.11.0 milestone Dec 23, 2022
andrewrk added a commit that referenced this issue Apr 14, 2023
Now they use slices or array pointers with any element type instead of
requiring byte pointers.

This is a breaking enhancement to the language.

The safety check for overlapping pointers will be implemented in a
future commit.

closes #14040
andrewrk added a commit that referenced this issue Apr 14, 2023
Now they use slices or array pointers with any element type instead of
requiring byte pointers.

This is a breaking enhancement to the language.

The safety check for overlapping pointers will be implemented in a
future commit.

closes #14040
@matklad
Copy link
Contributor

matklad commented Apr 14, 2023

It gets sliced from 0 to the same length as dest, triggering the same set of safety checks and possible compile errors as src[0..dest.len] would.

FWIW, in Tigerbeetle we have separate copy(.exact) and copy(.inexact) versions to control the slicing behavior, to capture the cases where you think the lengths are the same, but they are not due to a bug, and to, ahem, communicate intention precisely.

https://github.com/tigerbeetledb/tigerbeetle/blob/095338b3fbbe8f6a1a3a6253c94d1029f063f373/src/stdx.zig#L100

@andrewrk
Copy link
Member Author

Thanks for the insight, @matklad. I put some more thought into it and decided to make it the "exact" behavior. So, @memcpy will be consistent with for loops, in the sense that the source and destination iterables will be peers, rather than one of them dictating the minimum length of the other. Either one may provide a length, but at least one must provide a length. If both provide a length, it asserts that they are equal (just like for loops).

andrewrk added a commit that referenced this issue Apr 22, 2023
Now they use slices or array pointers with any element type instead of
requiring byte pointers.

This is a breaking enhancement to the language.

The safety check for overlapping pointers will be implemented in a
future commit.

closes #14040
andrewrk added a commit that referenced this issue Apr 23, 2023
Now they use slices or array pointers with any element type instead of
requiring byte pointers.

This is a breaking enhancement to the language.

The safety check for overlapping pointers will be implemented in a
future commit.

closes #14040
andrewrk added a commit that referenced this issue Apr 24, 2023
Now they use slices or array pointers with any element type instead of
requiring byte pointers.

This is a breaking enhancement to the language.

The safety check for overlapping pointers will be implemented in a
future commit.

closes #14040
andrewrk added a commit that referenced this issue Apr 25, 2023
Now they use slices or array pointers with any element type instead of
requiring byte pointers.

This is a breaking enhancement to the language.

The safety check for overlapping pointers will be implemented in a
future commit.

closes #14040
andrewrk added a commit that referenced this issue Apr 25, 2023
Now they use slices or array pointers with any element type instead of
requiring byte pointers.

This is a breaking enhancement to the language.

The safety check for overlapping pointers will be implemented in a
future commit.

closes #14040
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted This proposal is planned. breaking Implementing this issue could cause existing code to no longer compile or have different behavior. enhancement Solving this issue will likely involve adding new logic or components to the codebase. proposal This issue suggests modifications. If it also has the "accepted" label then it is planned.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants