Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Safe memcpy for slices ([T]::copy_from_slice) #1419

Merged
merged 12 commits into from
Feb 18, 2016
Merged

Safe memcpy for slices ([T]::copy_from_slice) #1419

merged 12 commits into from
Feb 18, 2016

Conversation

strega-nil
Copy link

No description provided.

@Aatch
Copy link
Contributor

Aatch commented Dec 20, 2015

Looks good to me. I'd personally prefer only panicking on src.len() > dst.len() but slicing dst to the appropriate length isn't difficult, so I don't feel strongly about it.

@bluss
Copy link
Member

bluss commented Dec 20, 2015

I think it's a good call to have a free function for std::slice::copy(src, dst), the two parameters have equal importance.

@strega-nil
Copy link
Author

@Aatch I also don't feel too strongly about this, but the current design does make this specific use nice:

// let src: &[T];
let mut dst: [T; N] = unsafe { std::mem::uninitialized() };
std::slice::copy(src, &mut dst); // You know that dst is fully initialized from here on

@Dr-Emann
Copy link

As a note, dlang has special syntax for set and copy, and it only allows arrays of equal size.

e.g.

auto x = new int[10];
auto y = new int[11];
x[] = 5; // fill x with 5's
y[] = x[]; // Array lengths don't match for copy: 10 != 11

However, `std::slice::bytes::copy_memory`, the function I'm basing this on, only
panics if `dst.len() < src.len()`. So... room for discussion, here.

These are necessary functions, in the opinion of the author.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They could be methods on slices, not necessarily free functions.

@Amanieu
Copy link
Member

Amanieu commented Dec 20, 2015

I think fill would be a better name than set. This matches the C++ std::fill function which fills a range with copies of a single.

@eddyb
Copy link
Member

eddyb commented Dec 20, 2015

@Amanieu I definitely prefer xs.fill(x). Not sure if there are actual issues around adding methods to slice types.

@strega-nil
Copy link
Author

@eddyb @Amanieu I don't think there are issues around adding methods to slice types, but I don't personally like having these functions as methods. These feel "right" as free functions (at least, copy does). I can see set as a method called fill however.

Although, perhaps fill would be better as a method for Clone types. This is meant to be lowered to a memset in most cases, or something similar.

@eddyb
Copy link
Member

eddyb commented Dec 20, 2015

@ubsan Keep in mind that the main reason for std::slice and std::str (and all of the modules for integers and floats, but I digress) is that once upon a time, there were no inherent impls for primitives, so you had to choose between free functions and traits.
And free functions won for a long time.

I can see how copy looks cleaner as a free function, but I think assignment syntax is optimal there.

@strega-nil
Copy link
Author

@eddyb true, for set. I think I'll change the RFC to be fn set -> fn [T]::fill. I still think copy looks better as a free function, however.

specifically about being defined for slices with uninitialized values

Also add some language about alternatives
Add one function to `std::slice`.

```rust
pub fn copy<T: Copy>(src: &[T], dst: &mut [T]);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this cause issue when T contains &U? We have to be careful to avoid lifetime lengthening. But I guess variance will fix it.

@Manishearth
Copy link
Member

Note that free functions are harder to discover doc-wise. The general rust community seems to avoid them in code; though I'm not certain of this.

@strega-nil
Copy link
Author

@Manishearth copy, imho, is better as a free function. copy_from and copy_to are both valid ways of writing copy as a method, and are both fairly ugly to write in regular code, imo.

slice.copy(slice2); // which is src? which is dst?
slice.copy_to(slice2); // okay, it's obvious here, but it's kind of ugly
slice.copy_from(slice2); // same as ^
std::slice::copy(slice, slice2); // nice! also, a connection to std::ptr::copy_nonoverlapping here.

@Manishearth
Copy link
Member

I think copy_to or copy_from are better (I like copy_to better of these, but not for any real reason). They're clearer, don't obfuscate the src/dst, and are methods instead of free fns. copy(src, dst) will invariably have you looking up the docs all the time. The &[T]/&mut [T] distinction helps, but in case both happen to be mutable you're stuck again.

@jFransham
Copy link

@ubsan equally to your first example:

std::slice::copy(slice, slice2); // which is src? which is dst?

I think this is what @Manishearth was talking about when he mentioned looking up the docs. It's obvious if you've used memcpy for years but for people new to systems programming it will be confusing.

@Manishearth
Copy link
Member

It's obvious if you've used memcpy for years

Not even that, because that convention isn't uniformly followed across programming languages. I think even rust had a deliberately "backwards" memcpy ordering in the past (I think it got fixed in the rush for 1.0 or something)

@tomjakubowski
Copy link
Contributor

@ubsan equally to your first example:

std::slice::copy(slice, slice2); // which is src? which is dst?

I think this is what @Manishearth was talking about when he mentioned looking up the docs. It's obvious if you've used memcpy for years but for people new to systems programming it will be confusing.

The order proposed in the RFC is the opposite of memcpy's, in fact:

void *memcpy(void *restrict dst, const void *restrict src, size_t n);

Anyway, in many cases the Rust would look more like:

std::slice::copy(&slice, &mut slice2);

which makes it pretty clear which argument is source and which is destination.

```

`fill` loops through slice, setting each member to value. This will lower to a
memset in all possible cases. It is defined to call `fill` on a slice which has
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it might be worth mentioning how the behavior is defined :)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"defined" meaning "will not exhibit undefined behavior". :P

@strega-nil strega-nil changed the title Safe memcpy, memset for slices (std::slice::{ copy, set }) Safe memcpy, memset for slices [T]::{ copy_from, fill } Dec 20, 2015
@strega-nil strega-nil changed the title Safe memcpy, memset for slices [T]::{ copy_from, fill } Safe memcpy, memset for slices ([T]::{ copy_from, fill }) Dec 20, 2015
@strega-nil
Copy link
Author

@Manishearth @tomjakubowski I polled some people in the rust community, and it seems that generally people don't like my free functions, and they've convinced me, so I've decided to switch to methods. This also means that I've switched to the name copy_from. Is there anything else that you see?

@Amanieu
Copy link
Member

Amanieu commented Dec 20, 2015

Some alternatives that might be worth considering:

  1. Allow direct assignment of slices with the same panic semantics as the proposed copy function:
a[..] = b[..];

This is currently disallowed because [T] is an unsized type.

  1. Make fill an iterator method. This would allow it to be used on any collection type instead of just slices. This method would only be available when Iter::Item is &mut T.

  2. Allow any Clone type for copy and fill instead of only Copy types. This would make the functions more useful for generic code.

  3. Maybe consider the general concept of copying data from one iterator to another. This would allow values to be modified prior to copying by using map for example:

iter::copy(b.iter(), a.iter_mut());
iter::copy(b.iter().map(|x| x * 2), a.iter_mut());

The downside is that this might be getting too generic, and it wouldn't make sense to have the proposed panic semantics when the iterator lengths don't match.

@Manishearth
Copy link
Member

As far as iterators go there's not much value (relatively) there because you can write this as a function already.

The reason this RfC is important is that currently there's no way to copy/fill slices in safe Rust efficiently. You have to use a for loop and hope the optimizer magicks it. This RfC provides a way to do that. For iterators, a one-line for loop just works and can't be optimized further, so there's much less value there.

@Manishearth
Copy link
Member

@ubsan mostly looks good, 👍 for this rfc.

Small nit: Perhaps fill() should also have a descriptive name (fill_from in its current form)?

@strega-nil
Copy link
Author

@Manishearth Hmm, I can see your argument... I do like @eddyb's suggestion of fill_with. Are there any other suggestions?

@bluss
Copy link
Member

bluss commented Jan 22, 2016

The naive version of the "clone_from_slice" loop produces terrible code and the naive fill loop produces good code. It's good that we have clone_from_slice so that the right way to tickle llvm for an optimized copying is abstracted out into a method. I don't think the pragmatic argument is the only one for clone_from_slice though.

I think .fill(x) is a nice API, but it's not as important as .copy_from().

@strega-nil
Copy link
Author

Agreed, it isn't as important. The only reason I also added fill is because memset and memcpy kinda go together. copy_from is the important part of this RFC.

@alexcrichton
Copy link
Member

In the case that fill was a bit of an addition, perhaps that API can be left to its own RFC? (or perhaps an RFC issue for now) It sounds like there's some outstanding questions around it and otherwise copy_from seems fine modulo naming.

@strega-nil
Copy link
Author

@alexcrichton I would be fine with that. I don't personally see any pressing need for it, and no one else seems to see any pressing need for it.

@strega-nil strega-nil changed the title Safe memcpy, memset for slices ([T]::{ copy_from, fill }) Safe memcpy for slices ([T]::copy_from) Jan 27, 2016
@strega-nil
Copy link
Author

Yeah, I'm taking out [T]::fill. It can be decided upon later. The important part of this RFC is [T]::copy_from.

@BurntSushi
Copy link
Member

👍 to this RFC. Guaranteed memcpy is good. I do also think it should be named copy_from_slice to be consistent with clone_from_slice.

@alexcrichton
Copy link
Member

🔔 This RFC is now entering its week-long final comment period 🔔


My personal opinion is that this is good to go with the name copy_from_slice, the semantics all seem good to me though!

@alexcrichton alexcrichton added the final-comment-period Will be merged/postponed/closed in ~10 calendar days unless new substational objections are raised. label Feb 11, 2016
@strega-nil
Copy link
Author

@alexcrichton I don't personally like the name, because it's a very common function (at least in my personal code) to have such a long name. However, I do see the benefit of it in terms of consistency, so I am not opposed to it. I'd just prefer a shorter name.

@bluss
Copy link
Member

bluss commented Feb 12, 2016

Let's get this in. Thanks for pushing it @ubsan (and I like your new avatar).

Using this to replace most uses of clone_from_slice in rustc and libstd will improve performance for debug builds by using memcpy directly, so we benefit immediately.

@strega-nil
Copy link
Author

@bluss thanks :) I needed a new avatar very badly...

Yeah, I will be interested in io::Write performance on Debug (I think it uses clone_from_slice currently).

@aturon
Copy link
Member

aturon commented Feb 12, 2016

A couple thoughts:

  • To clarify some points that @bluss made: the compiler and lang teams were against using a compiler hack to do custom specialization for clone_from_slice. But the possibility of doing so through specialization proper is definitely open (and specialization is likely to land in the near future).
    • The main question in actually specializing clone_from_slice for Copy data is the divergence currently possible -- the fact that you can write custom Clone impls for Copy data. This was necessary for various reasons, and would be difficult to change backwards-compatibly. But we could viably consider this a contract violation, and still give clone_from_slice memcpy semantics on Copy data.
  • On the other hand, there is perhaps some value to having a separate method which is purely a safe memcpy, and in particular asserts the Copy-ness of the data -- even if we also specialize clone_from_slice. I personally could go either way on that question.
  • Finally, I agree with the general sentiment that the name of this method should match the idiom for clone_from_slice, which was recently stabilized.

@strega-nil strega-nil changed the title Safe memcpy for slices ([T]::copy_from) Safe memcpy for slices ([T]::copy_from_slice) Feb 13, 2016
@alexcrichton
Copy link
Member

The libs team discussed this RFC during triage yesterday, and the conclusion was to merge this RFC. We discussed the possibility of having clone_from_slice guarantee the same semantics as copy_from_slice here if T is Copy (disregarding possible diverging implementations of copy/clone), but our conclusion was that even if we had that behavior we would still want to have copy_from_slice. This method is a good static assertion that the operation is indeed a memcpy and we're also not sure how comfortable we would be relying on specialization for semantics like that.

Anyway, I shall merge soon. Thanks again for the RFC @ubsan!

@alexcrichton alexcrichton merged commit ea0ad1c into rust-lang:master Feb 18, 2016
bors added a commit to rust-lang/rust that referenced this pull request Feb 26, 2016
@ghost
Copy link

ghost commented Mar 4, 2016

FWIW at this point, I'd most definitely prefer the shorter name copy_from despite inconsistency with clone_from_slice.

@Centril Centril added the A-slice Slice related proposals & ideas label Nov 23, 2018
Dylan-DPC-zz pushed a commit to Dylan-DPC-zz/rust that referenced this pull request Nov 19, 2020
…r=Mark-Simulacrum

Add the "memcpy" doc alias to slice::copy_from_slice

[RFC1419](rust-lang/rfcs#1419) describes `slice::copy_from_slice` as a "safe memcpy". This enables people searching for `memcpy` to find the `slice::copy_from_slice` method. Thanks!

## Screenshots

This is currently the output when searching for "memcpy" -- `copy_from_slice` is safe, and should be part of this list.

![Screenshot_2020-11-19 Results for memcpy - Rust](https://user-images.githubusercontent.com/2467194/99722964-c9e8fe80-2ab1-11eb-82a5-4afe703a0eea.png)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-slice Slice related proposals & ideas final-comment-period Will be merged/postponed/closed in ~10 calendar days unless new substational objections are raised. T-libs-api Relevant to the library API team, which will review and decide on the RFC.
Projects
None yet
Development

Successfully merging this pull request may close these issues.