Tracking issue for `push_all` stabilization

[aturon]

The Vec::push_all method special-cases pushing a slice onto a vector, and historically has performed much better than using extend (depending on how LLVM optimizations are working out).

We need to offer a method with maximal performance somehow, but ideally this would come from extend, either through optimization or through impl specialization.

[jonas-schievink]

cc #26902

[Gankro]

Unless specialization lands soon, I'm tempted to just stabilize this as "this will be deprecated when it's no longer necessary".

[jnordwick]

Have the IR (lack of) hinting issues been worked out?

[sfackler]

Nominating for 1.6 discussion.

[alexcrichton]

🔔 This issue is now entering its cycle-long final comment period for stabilization 🔔

The conclusion of the libs team is that we can stabilize this and just deprecate it once specialization lands and makes this function obsolete.

[cuviper]

IMO, planned deprecation defeats the point of stabilizing it in the first place. i.e. I still won't want to use it if it's definitely going to be yanked out later...

[sfackler]

It's not going to be yanked out ever if stabilized. It will eventually be deprecated in favor of extend, but remain in place.

[cuviper]

Ok, I thought deprecation was more aggressive than that. No problem then.

[briansmith]

IMO, planned deprecation defeats the point of stabilizing it in the first place. i.e. I still won't want to use it if it's definitely going to be yanked out later...

I agree with this.

It's not going to be yanked out ever if stabilized. It will eventually be deprecated in favor of extend, but remain in place.

IMO, that would be a poor choice.

In the discussion on clone_from_slice, there seemed to be a lot of agreement that the compiler could hard-code some specializations for features that are urgently needed so that they're not blocked on generalized specialization. This seems like a candidate for that kind of treatment.

One of the main reasons for moving ahead with clone_from_slice is that clone_from_slice can be changed to have specific performance guarantees. push_all could be like that, but currently it is not. Unless push_all is improved to have similar performance guarantees, I don't think it should be stabilized.

Conversely, if useful performance guarantees are added to it (at most one resizing of the array, at most one bounds check on each array, equivalent to set_len followed by copy_nonoverlapping for POD/Copy types), then I do think it makes sense to stabilize it since it seems like it would be a long time before extend would be able to guarantee those things.

It is also unfortunate that the name push_all is so different from clone_from_slice. It might be good to rename it push_from_slice, or a different name if clone_from_slice gets renamed.

[erickt]

Would it be worth considering another name for this method? I find push_all slightly ambiguous with respect to extend. Perhaps push_slice would be a bit more clear? It'd blend a bit better with clone_from_slice.

[briansmith]

I propose renaming this function to extend_from_slice, if it is going to be stabilized at all. See #27750 (comment) for the rationale.

[aturon]

@briansmith

In the discussion on clone_from_slice, there seemed to be a lot of agreement that the compiler could hard-code some specializations for features that are urgently needed so that they're not blocked on generalized specialization. This seems like a candidate for that kind of treatment.

I don't think there's widespread agreement about adding ad hoc specializations/intrinsics, barring very exceptional circumstances. But @rust-lang/lang and @rust-lang/compiler may want to weigh in.

One of the main reasons for moving ahead with clone_from_slice is that clone_from_slice can be changed to have specific performance guarantees. push_all could be like that, but currently it is not. Unless push_all is improved to have similar performance guarantees, I don't think it should be stabilized.

Conversely, if useful performance guarantees are added to it (at most one resizing of the array, at most one bounds check on each array, equivalent to set_len followed by copy_nonoverlapping for POD/Copy types), then I do think it makes sense to stabilize it since it seems like it would be a long time before extend would be able to guarantee those things.

I'm a little confused about the overall thrust here. To be clear, the reason that push_all hasn't been stabilized in the past has not been a desire to improve its implementation, but rather to specialize extend to use the push_all implementation. In other words, the point of dissatisfaction was having semantically identical APIs where the client is forced to explicitly call the more specialized one for the best performance.

Now, that said, i think the performance guarantees you're raising are an important (though orthogonal) point. One uncomfortable thing about the current push_all implementation is that it's carefully tailored to trigger the right LLVM optimizations. Of course, it'd be much nicer if we could provide an actual guarantee about the generated code instead -- and not just for this function, but for a handful of others throughout std.

One way to address this desire might be to add more ad hoc intrinsics, but I wonder if there's any way we might eventually allow "inline MIR", so to speak, and whether that'd be enough to express the precise code you want to generate.

On the whole, I still think it's reasonable to stabilize something push_all as the best way we currently have of appending a slice onto a vector. I'm not particularly bothered by the intent to deprecate, both because it will be some time before we're in a position to do so, and because the whole point of the stabilization and deprecation process is to allow relatively smooth evolution without breakage (since deprecated items are not actually removed). We shouldn't let the perfect be the enemy of the good, especially when we have a good process for iterating toward perfection over time :)

It is also unfortunate that the name push_all is so different from clone_from_slice. It might be good to rename it push_from_slice, or a different name if clone_from_slice gets renamed.

@alexcrichton didn't mention this, but no one was particularly a fan of the name. 👍 on renaming, and matching the clone_from_slice idiom probably makes sense.

[Gankro]

👍 unhappy_no_specialization_extend_slice_effecient

[nrc]

I don't think there's widespread agreement about adding ad hoc specializations/intrinsics

+1000 to not adding an ad hoc hack here.

However, I'd love to see some kind of push_all stabilised - it's probably the unstable function I use the most.

[bluss]

I don't think there's widespread agreement about adding ad hoc specializations/intrinsics, barring very exceptional circumstances.

Absolutely, I would say the same thing: it's very unlikely there is any agreement about it. It's just my crazy idea. 😄

[nikomatsakis]

On Mon, Nov 09, 2015 at 01:04:45PM -0800, Nick Cameron wrote:

I don't think there's widespread agreement about adding ad hoc specializations/intrinsics

+1000 to not adding an ad hoc hack here.

and +1K more

[bluss]

Notice that the std::io::Write's write method for Vec<u8> is already a stable way to get hold of push_all when it comes to u8 elements, arguably the most important case.

[alexcrichton]

The libs team discussed this during triage today and the decision was to stabilize while renaming the method to extend_from_slice.