overloaded-`box` and placement-`in`

[pnkfelix]

Summary:

• Change placement-new syntax from: box (<place-expr>) <expr> instead to: in <place-expr> { <block> } (Note: was in (<place-expr>) <expr> previously)
• Change box <expr> to an overloaded operator that chooses its implementation based on the expected type.
• Use unstable traits in core::ops for both operators, so that libstd can provide support for the overloaded operators; the traits are unstable so that the language designers are free to revise the underlying protocol in the future post 1.0.

(rendered text/)

(rendered draft)

Tracking issue: rust-lang/rust#22181

[nikomatsakis]

👍 I am happy with this compromise (stable intentions, unstable protocol).

[alexcrichton]

To clarify, does this affect inference in an adverse way if we add later implementations to support box expressions creating Rc<T> and Arc<T>? For example, would this compile successfully today with only one implementation, but fail with many implementations?

drop(box 3);

If we only have one implementation of boxing, does this mean that we'll successfully resolve this to Box<i32> vs Rc<i32>?

[pnkfelix]

That's a good question; I had assumed that the coherence rules would ensure that drop(box 3) would be rejected, but its possible I misremember/misunderstand.

[alexcrichton]

I think it will too, I just haven't kept up with all the coherence rules and trait selection recently! FWIW this does not compile:

pub trait Foo {
    fn make() -> Self;
}

impl Foo for i32 {
    fn make() -> i32 { 2 }
}

fn main() {
    drop(Foo::make());
}

foo.rs:10:5: 10:9 error: unable to infer enough type information about `_`; type annotations required [E0282]
foo.rs:10     drop(Foo::make());
              ^~~~
error: aborting due to previous error

[pnkfelix]

(in the prototype from the Appendix, if you add

    drop(box_!( 7 ));

it fails to compile, with the message

<anon>:36:13: 36:22 error: unable to infer enough type information about `_`; type annotations required [E0282]
<anon>:36         let mut place = ::protocol::BoxPlace::make_place();
                      ^~~~~~~~~

So, a terrible error message, but that does jibe with my hypothesis that such code would be rejected.

[alexcrichton]

Ok, sounds good to me!

[nikomatsakis]

@alexcrichton I am assuming we will add implementations for Rc and Arc once the protocol exists, so this concern is somewhat moot.

[brson]

Will this land behind a feature gate?

[eddyb]

I prefer in(v.back()) 3, fwiw, as "emplace" is just a longer way of conveying the same thing in does.
Also it could return &mut T, but I may be mistaken.

This is straying a bit from the topic at hand, and I'm not sure what will come of it, but I have the start of a design that makes *v.back() = 3 possible.

[pnkfelix]

Regarding names and return types: The beauty of the approach of this RFC is to try to avoid spending time right now quibbling over these kinds of details, which are the topic of library design and not language design. (Having said that, I am not attached to any particular names here.)

The reason I said "has return type () was mostly to make it clear that the API can be made flexible enough so that the in (<place-expr>) <expr> form is usable even in scenarios where it is solely used for imperative update and not for its return value.

---

Regarding making *v.back() = 3 possible: Okay, that's good, but I do not think such a feature will make the need for placement-in obsolete. (It would, I guess, obsolete its use in this example of Vec::emplace_back.)

[pnkfelix]

@brson asked:

Will this land behind a feature gate?

Hmm. So my gut reaction is "yes; shouldn't everything land behind a feature gate?"

But another (more important) way to interpret your question is as follows:

• Assuming this lands in time for the first beta release, will this be feature-gated during the first beta cycle, or will we attempt to unfeature-gate it in time for the first beta?
• Follow-on Q: If we do have it feature gated during the first beta cycle, will we ensure that there is at least one beta cycle with it unfeature-gated?

(Where "this", I believe, refers to the box <expr> syntax and the in (<place-expr>) <expr> syntax.)

I would like to try to ensure we have a beta cycle with the syntax unfeature-gated. Otherwise, its not clear why we would attempt to put it into the 1.0 release at all.

[netvl]

Has anyone considered using box <expr> in <place> syntax, with in <place> part being optional?

[pnkfelix]

@netvl a large host of syntaxes have been considered. I guess I should have referenced those discussions in the RFC.

See e.g. https://github.com/pnkfelix/rfcs/blob/fsk-placement-box-rfc/text/0000-placement-box.md#same-semantics-but-different-surface-syntax which does list box <expr> in <place-expr> as one of the options.

One particular constraint I am interested in maintaining is: The evaluation order should reflect the order of expressions in the syntax.

Also the author of #405 points out another drawback: the <expr> is usually longer than <place-expr>, so it is often better for the <expr> to be on the right-hand-side.

(Meanwhile, the other alternative of in <place-expr> box <expr> is ambiguous in general, I think. But of course that is not what you suggested.)

[nikomatsakis]

@netvl another problem with syntaxes that have an optional suffix is that they tend to induce ambiguity. For example, box foo + box bar in place -- is that box (foo + (box bar in place)) or box (foo + box bar) in place, and so forth. (The current high precedence of box might make this a non-issue, but I think that precedence is itself a bug.)

[kennytm]

👍 on the semantic, but 👎 on the placement in syntax.

---

<bikeshed>

While in (place) value is syntactically unambiguous, it looks completely unnatural as a statement alone, mainly because there are no verbs in the correct place, and also using in alone is usually associated with iteration (for x in y) and member testing (elem in set).

in (v.emplace_back()) 3;
// feels like, what? "In... emplace to v's back, 3. ...?"
// looks like a strange version of `v.emplace_back(3);`?
// then why can't we use `v.emplace_back(3);` etc. etc.

let b6 = in (HEAP) 6;
// feels like, check if 6 is in the HEAP?

Just some alternatives I feel more natural, if the order is required to be placer → expression:

// use `box in` phrase. that is, `box expression in[to] place`.
box in (v.end()) 1 + 2 + 3;
let b6 = box in (HEAP) 4 + 5 + 6;

// more rearrangements. already mentioned by pnkfelix.
// the `box` keyword is *required* for human to get the correct context.
in v.end() box 1 + 2 + 3;
let b6 = in HEAP box 4 + 5 + 6;

// reuse `move` keyword. should have no conflict with `move |x| y + x`.
// but sounds like going to move the placer instead of the expression.
// misnomer since nothing is actually moved.
move (v.end()) 1 + 2 + 3;
let b6 = move (HEAP) 4 + 5 + 6;

// use `move in` phrase. a bit too verbose, but make it clear that the placer
// is the target. rearranged from `move expression in[to] place`.
move in (v.end()) 1 + 2 + 3;
let b6 = move in (HEAP) 4 + 5 + 6;

// use a new operator. free to define whatever it means, but hard to learn about it.
v.end() <| 1 + 2 + 3;
let b6 = HEAP <| 4 + 5 + 6;
v.end() <~ 1 + 2 + 3;
let b6 = HEAP <~ 4 + 5 + 6;

// just create another built-in syntax extension? 
emplace!(v.end(), 1 + 2 + 3);
let b6 = emplace!(HEAP, 4 + 5 + 6);

</bikeshed>

[liigo]

+1! Better than this rfc.
I prefer box in x y or in x box y.
On Feb 6, 2015 12:02 AM, "kennytm" notifications@github.com wrote:

[image: 👍] on the semantic, but [image: 👎] on the placement in

syntax.

While in (place) value is syntactically unambiguous, it looks completely
unnatural as a statement alone, mainly because there are no verbs in
the correct place, and also using in alone is usually associated with
iteration (for x in y) and member testing (elem in set).

in (v.emplace_back()) 3;// feels like, what? "In... emplace to v's back, 3. ...?"// looks like a strange version of v.emplace_back(3);?// then why can't we use v.emplace_back(3); etc. etc.
let b6 = in (HEAP) 6;// feels like, check if 6 is in the HEAP?

Just some alternatives I feel more natural, if the order is required to be
placer → expression:

// use box in phrase. that is, box expression in[to] place.box in (v.end()) 1 + 2 + 3;let b6 = box in (HEAP) 4 + 5 + 6;
// more rearrangements. already mentioned by pnkfelix.// the box keyword is required for human to get the correct context.in v.end() box 1 + 2 + 3;let b6 = in HEAP box 4 + 5 + 6;
// reuse move keyword. should have no conflict with move |x| y + x.// but sounds like going to move the placer instead of the expression.// misnomer since nothing is actually moved.
move (v.end()) 1 + 2 + 3;let b6 = move (HEAP) 4 + 5 + 6;
// use move in phrase. a bit too verbose, but make it clear that the placer// is the target. rearranged from move expression in[to] place.
move in (v.end()) 1 + 2 + 3;let b6 = move in (HEAP) 4 + 5 + 6;
// use a new operator. free to define whatever it means, but hard to learn about it.
v.end() <| 1 + 2 + 3;let b6 = HEAP <| 4 + 5 + 6;
v.end() <~ 1 + 2 + 3;let b6 = HEAP <~ 4 + 5 + 6;
// just create another built-in syntax extension?
emplace!(v.end(), 1 + 2 + 3);let b6 = emplace!(HEAP, 4 + 5 + 6);

—
Reply to this email directly or view it on GitHub
#809 (comment).

[nikomatsakis]

@brson @pnkfelix I wonder if it makes sense to discuss box expr and in(place) expr separately. That is, I could imagine feature-gating the latter, with which we've had no experience, but accepting the former.

[nikomatsakis]

@liigo @kennytm I think the syntax would have to be box(in place) expr. In general, placing two expressions end-to-end with no separator is asking for ambiguities like box in place -5 (is that box in (place - 5)?)

[nikomatsakis]

that said, while I agree that box(in place) 5 reads a bit more like English, it seems like extra tokens for little benefit. Also, the verb box isn't always appropriate. Consider: box(in vec.end()) value. Here we are just writing value into the vector, not "boxing" it in any normal sense of the term. C++ happens to overload new for this purpose, but it's not clear that we should do the same.

[nikomatsakis]

One more thought regarding stabilization -- even if we want to land this feature-gated, I think it'd be better to take the RFC and get this work in tree. I feel pretty sure that we want this sort of "emplacement" feature, even if we're not ready to commit to the details in particular. I'd rather have it in tree so we can experiment and gain experience with it. (I'd feel differently if no work had been done -- but @pnkfelix has most of the pieces implemented by now.)

[glaebhoerl]

I agree that inference-based overloaded box seems like the right way to go. Could you explain how type inference might work in a case like let x: Rc<_> = box format!("Hello"); from the RFC? In particular, how can it infer the element type (the _) separately from the Rc? It's not obvious to me how this might work without HKT.

And I may be misunderstanding something, but why add the in syntax now if, as of yet, it would have no actual uses/users?

[kennytm]

@glaebhoerl

Could you explain how type inference might work in a case like let x: Rc<_> = box format!("Hello"); from the RFC?

I believe it's no different from how this works:

let v: Vec<_> = (0u32 .. 10).collect();

---

@nikomatsakis

@liigo @kennytm I think the syntax would have to be box(in place) expr. In general, placing two expressions end-to-end with no separator is asking for ambiguities like box in place -5 (is that box in (place - 5)?)

I agree, that's why I suggested box in (place) expr with a pair of mandatory parenthesis like the RFC, not just box in place expr.

that said, while I agree that box(in place) 5 reads a bit more like English, it seems like extra tokens for little benefit. Also, the verb box isn't always appropriate. Consider: box(in vec.end()) value. Here we are just writing value into the vector, not "boxing" it in any normal sense of the term. C++ happens to overload new for this purpose, but it's not clear that we should do the same.

I also agree that box is not a precise verb for "in-place initialization", but there aren't many verbs to choose from the list of keywords and reserved words 😞 I would prefer a macro syntax emplace!(place, expr) to in(place) expr if we don't use box or move and the former is possible.

[eddyb]

@nikomatsakis I agree that in can be left for later - see my *v.back() = x; example - it might not even require linear types, though it would need some overloadable post-store behavior, i.e. finalize from this RFC.
But I think we need box now and having the machinery for in won't hurt as long as it's unstable/feature-gated.

[petrochenkov]

A note about emplace-like functionality:
As C++11 experience has shown, when it's available, it will become the default method of inserting elements in containers, since it's never performing worse than "normal insertion" and is often better. So it should really have as short and convenient syntax as possible.

[petrochenkov]

It'd be very nice if chained NRVO, URVO and placement-new were guaranteed to apply in scenarios like

struct Big {
    data: [u8; 100]
}

fn constructor_function1() -> Big {
    Big { data: [37; 100] }
}

fn constructor_function2() -> Big {
    let mut result = constructor_function1();
    // ... modify result
    result;
}

fn main() {
    let v = Vec::new();
    in v.back() constructor_function2();
}

and elide all the unnecessary copies.

[pnkfelix]

@petrochenkov regarding return-value-optimization: the intention is that the desugaring described in the RFC hopefully presents the code in a manner where LLVM can optimize it accordingly. If we observe that LLVM fails to optimize the code as desired, then we can instead switch from a macro-based implementation to more integrated support within the whole compiler pipeline; but that will hopefully be unnecessary.

(i.e., the focus of this RFC is on the changes that are visible to the programmer, not on the quality of implementation, though of course we do care about quality of implementation.)

[pnkfelix]

I just wanted to jot a few cases where the current desugaring still fails to do as well as I would like, even after augmenting rustc with rust-lang/rust#22012 :

    let b8: Box<[i32]> = box_!( [] );
    println!("b8: {:?}", b8);

    trait Invoke<A=(),R=()> { fn invoke(self: Box<Self>, arg: A) -> R; }
    impl<A,R,F> Invoke<A,R> for F where F : FnOnce(A) -> R
    {
        fn invoke(self: Box<F>, arg: A) -> R { let f = *self; f(arg) }
    }

    fn with_arg<A,R,F>(f: F) -> Box<Invoke<A,R>+Send> where
        F: FnOnce(A) -> R, F: Send { box_!(f) }
    let b9 = with_arg(|x| x+1);
    println!("b9.invoke(8): {}", b9.invoke(8));

(Both of the above cases compile fine if you use box <expr> instead of box_!(<expr>). And they are both used in contexts that expect a Box<T> from the expression (for some T), though admittedly it is an unsized T in both cases.)

It would be good to resolve that, or at least incorporate it into the drawbacks section if I cannot fix it quickly.

---

Some of the errors I'm seeing from the desugaring on the above cases is arising because the traits as written are not as general as they could be, with respect to the ?Sized generalization marker. But I think even after those problems are addressed, there is still likely to be an issue here.

---

It may be possible to generalize the trick I used in rust-lang/rust#22006, instead of making it a special case just for the as operator. But I do not know, it seems like a weird thing to try to e.g. build into the way we unify in general.

[liigo]

nice！
On Feb 6, 2015 1:36 AM, "Niko Matsakis" notifications@github.com wrote:

@liigo https://github.com/liigo @kennytm https://github.com/kennytm I
think the syntax would have to be box(in place) expr. In general, placing
two expressions end-to-end with no separate is asking for ambiguities like box
in place -5 (is that box in (place - 5)?)

—
Reply to this email directly or view it on GitHub
#809 (comment).

[nikomatsakis]

The core team has decided to accept this RFC. However, the box and in expressions described here will remain under feature gate for the time being (and they may be unfeature-gated separately). In particular, it is unlikely that we will unfeature-gate them for 1.0.

The reason for this feature gate is not only the usual matter of not wanting to expose partially implemented features, but also some questions as to the desirability of the overall design (as raised in this discussion thread):

1. Can we improve inference related to box results and implicit coercions?
2. What syntax to use for in(place) expr / in place { expr }? There have been...numerous suggestions. ;)
3. Do we need in at all, or can we replace it with some future possible feature such as DerefSet or &out etc?
   • Note that neither of these features is accepted nor fully thought out, and this condition is not meant to serve as an endorsement of either feature.
4. Do we want to improve the protocol in some way?
   • Support for DST expressions such as box [22, ..count] (where count is a dynamic value)?
   • Protocol making use of more advanced language features?

Ordinarily these uncertainties might lead to the RFC being postponed. However, given that there is an existing box operator which is widely used in the compiler, servo, and elsewhere, we've decided to accept it instead, as the RFC represents an improvement on the current state and points in a direction we consider promising. Accepting the RFC (and landing @pnkfelix's interim implementation work) will thus give us immediate experience with the design and help in determining the best final solution.

[nikomatsakis]

Tracking issue is rust-lang/rust#22181

[nikomatsakis]

Up to date final text will be available here: https://github.com/rust-lang/rfcs/blob/master/text/0809-box-and-in-for-stdlib.md

[nwin]

in expr { expr } does look weird. It is the only syntax element that uses a preposition and the brackets seem to be superfluous. For consistency reasons I would urge to use at least one verb. This also makes the syntax unambiguous such that the brackets would not be necessary.

• in <place_expr> place <expr>
• in <place_expr> put <expr>
• put <expr> in < place_expr >
• move <expr> into < place_expr >
• place <expr> in < place_expr >

My favorite would be the latter variant because it does not use this strange inversion.

[diwic]

Do you think there'll be confusion if the name "box" can refer both to the "simple heap pointer" and "any type of pointer"? I e, if you write:

let x: Box<_> = box 3i;

A newcomer would probably expect something like this:

let x: Rc<_> = rc 3i;

Rather than:

let x: Rc<_> = box 3i; // confusing, is this a Rc smart pointer or a simple Box pointer?

In this case there is a distinction between Box meaning the simple pointer and box meaning something like "any type of container" but I suspect this can be a source of confusion, especially when talking (i e by human voice) about the language.

[nwin]

Box is more than just a simple heap pointer. It is a container like Rc. That it is optimized into a simple pointer, is just an implementation detail.

I don’t think this will lead to confusion, especially when talking about the language. Box, Rc and Arc are just different kinds of boxes and hence you can use box to place something in it.

[diwic]

Regardless of whether Box is simple or not, I mean there is room for confusion in the sentence "Box is a kind of box" :-)

[comex]

I agree that since emplacing is semantically almost always preferable (possibly faster, never slower, changed ordering of allocation and construction usually not significant), it should be as close to syntactically preferable as possible, or at least not much worse. *x.back() doesn't strike me as natural, but maybe I just need to get used to it. Requiring braces is a bad idea.

[kennytm]

cc RFC #98

[pnkfelix]

someone said that <- was already taken, but according to rust-lang/rust#22934, it seems like we might be able to use it as a legitimate token, i.e. PLACE <- EXPR (instead of the in PLACE { BLOCK } that was my previous attempt to win over the crowd here)

(of course one can say "but what about expressions like a<-b then being ambiguous with a < (-b)... sigh, i did try at one point to require spaces around all the comparison operators...)

[m13253]

(of course one can say "but what about expressions like a<-b then being ambiguous with a < (-b)... sigh, i did try at one point to require spaces around all the comparison operators...)

What about <=?

[kennytm]

@m13253 that's the less-than-or-equal operator...

[erkinalp]

What about <== ?