Undisambiguated generic arguments in expressions

[varkor]

Make disambiguating generic arguments in expressions with :: optional, allowing generic arguments to be specified without :: (making the "turbofish" notation no longer necessary).

Rendered

This makes the following valid syntax:

struct Nooper<T>(T);

impl<T> Nooper<T> {
    fn noop<U>(&self, _: U) {}
}

fn id<T>(t: T) -> T {
    t
}

fn main() {
    id<u32>(0u32); // ok
    let n = Nooper<&str>(":)"); // ok
    n.noop<()>(()); // ok
}

Thanks to @Centril, @scottmcm, @joshtriplett and @rpjohnst for feedback!

[Centril]

Some linebreaks would be real nice here (and elsewhere), whether at 80 characters or http://rhodesmill.org/brandon/2012/one-sentence-per-line/ :P

[Centril]

Some linebreaks would be real nice here (and elsewhere), whether at 80 characters or http://rhodesmill.org/brandon/2012/one-sentence-per-line/ :P

[varkor]

As soon as I start editing Markdown, I forget all about line breaks 😅

[Centril]

Some link as to why this step was taken would be nice here.

[Centril]

Some link as to why this step was taken would be nice here.

[Centril]

s/In order to/to/

[Centril]

s/In order to/to/

[Centril]

s/Obviously this/This/

[Centril]

s/Obviously this/This/

[xfix]

What about statements like this?

let a = (b<c, d>(e));

This can either mean a tuple

let a = ((b < c), (d > e));

Or a pair of generic arguments.

let a = b::<c, d>(e);

[xfix]

What about statements like this?

let a = (b<c, d>(e));

This can either mean a tuple

let a = ((b < c), (d > e));

Or a pair of generic arguments.

let a = b::<c, d>(e);

[varkor]

@xfix: it seems we had overlooked that ambiguity before, thanks for pointing it out. This looks pretty difficult to resolve to me, so I'm going to close the RFC unless someone has a resolution. It was a nice idea while it lasted 🙁

[varkor]

@xfix: it seems we had overlooked that ambiguity before, thanks for pointing it out. This looks pretty difficult to resolve to me, so I'm going to close the RFC unless someone has a resolution. It was a nice idea while it lasted 🙁

[eddyb]

@xfix Aaaaah, that's what I couldn't remember, which made this infeasible last time!
I should've asked @bstrie I guess, they might have remembered.

[eddyb]

@xfix Aaaaah, that's what I couldn't remember, which made this infeasible last time!
I should've asked @bstrie I guess, they might have remembered.

[Centril]

This was a sad day! 😭

[Centril]

This was a sad day! 😭

[varkor]

I've opened rust-lang/rust#53562 to ensure that any such attempt to unroot the Turbofish in the future shall have an earlier warning than we did.

[varkor]

I've opened rust-lang/rust#53562 to ensure that any such attempt to unroot the Turbofish in the future shall have an earlier warning than we did.

[novacrazy]

Additionally, at least for me personally, turbofish is less ambiguous for mentally parsing code as well, so I’ve never minded the slightly more verbose syntax. I also really don’t like this recent trend of oversimplification in Rust just to save a few characters...

[novacrazy]

Additionally, at least for me personally, turbofish is less ambiguous for mentally parsing code as well, so I’ve never minded the slightly more verbose syntax. I also really don’t like this recent trend of oversimplification in Rust just to save a few characters...

[varkor]

@novacrazy: I think you'll find that there's always been a trend of reducing characters in Rust (take a look at all the keywords) 😉
Regardless, the motivation here wasn't to reduce characters, but to make the syntax more consistent. I think you never get a syntax everyone agrees on, but in the case of turbofish, most of the feedback I've gathered has been positive towards making it unnecessary if possible.
It's a moot point, though.

[varkor]

@novacrazy: I think you'll find that there's always been a trend of reducing characters in Rust (take a look at all the keywords) 😉
Regardless, the motivation here wasn't to reduce characters, but to make the syntax more consistent. I think you never get a syntax everyone agrees on, but in the case of turbofish, most of the feedback I've gathered has been positive towards making it unnecessary if possible.
It's a moot point, though.

[eddyb]

@varkor Have you considered allowing only f<T>(), i.e. no commas?

[eddyb]

@varkor Have you considered allowing only f<T>(), i.e. no commas?

[Havvy]

@eddyb I feel like allowing it in the special case of zero or one type parameters is going to cause confusion and make the grammar that much more irregular.

[Havvy]

@eddyb I feel like allowing it in the special case of zero or one type parameters is going to cause confusion and make the grammar that much more irregular.

[BatmanAoD]

I'm actually with @novacrazy on this: I think the Rust syntax is superior to the C++ syntax. Angle brackets are pretty common; disambiguation is helpful.

[BatmanAoD]

I'm actually with @novacrazy on this: I think the Rust syntax is superior to the C++ syntax. Angle brackets are pretty common; disambiguation is helpful.

[Laaas]

This is really unfortunate, but I'd still really like a unification of providing generic arguments. The simplest way I see would just be to require turbofish everywhere, but this is quite ugly.

[Laaas]

This is really unfortunate, but I'd still really like a unification of providing generic arguments. The simplest way I see would just be to require turbofish everywhere, but this is quite ugly.

[artemshein]

I like the approach Kotlin takes on the disambiguation -- in case it's not obvious how to parse an expression, Kotlin will ask to put parens. No turbofish and no additional parens 99% of the time.

[artemshein]

I like the approach Kotlin takes on the disambiguation -- in case it's not obvious how to parse an expression, Kotlin will ask to put parens. No turbofish and no additional parens 99% of the time.

[comex]

Hmm… interesting. Kotlin doesn't have tuples (or a comma operator like C++), but you can get a similar effect with function calls:

foo(b<c, d>(e))

This appears to always be parsed as a generic, not a pair of comparisons – see this test program.

I suppose that if you did want two comparisons, you could add parentheses:

foo((b<c), d>(e))
    ^   ^

The approach Kotlin uses seems to require infinite lookahead. When encountering a <, the parser can't commit to whether to parse it as a comparison or as the start of a generic parameter list. For example, if the parser has seen this fragment of code so far:

foo(b<c,b<c,b<c,b<c,b<c,b<c,b<c, 

…these are all legal ways for the line to end:

1. foo(b<c,b<c,b<c,b<c,b<c,b<c,b<c, 4) (all comparisons)
2. foo(b<c,b<c,b<c,b<c,b<c,b<c,b<c, Int>>>>>>>()) (all generics)
3. foo(b<c,b<c,b<c,b<c,b<c,b<c,b<c, Int>>>()) (a mix of comparisons and generics!!)

(To play with this, look at lookahead.kt in the last link.)

That leaves me with a few questions:

• Are there peculiarities in Rust's grammar that would make adopting the same approach difficult or impossible?

• Was this approach ever considered in Rust's olden days, and rejected due to infinite lookahead or some other downside? (IMO, the benefit of killing turbofish would be well worth it.) Or did nobody realize it was a possibility?

• Adopting this in Rust would necessarily be a breaking change. Given (a<b, c>(d)), the 'default' parse must be as a generic rather than a comparison – because the comparison can be disambiguated by adding parentheses (((a<b), c>(d))) but there's nowhere to add parentheses that would make a generic the only valid interpretation. However, Rust currently does the opposite, defaulting to a comparison and requiring turbofish to force interpretation as a generic.

  Unfortunately, even if everyone suddenly agreed Rust should switch, AFAIK it would be way too late for the Rust 2018 edition. So… when's the next edition? :)

[comex]

Hmm… interesting. Kotlin doesn't have tuples (or a comma operator like C++), but you can get a similar effect with function calls:

foo(b<c, d>(e))

This appears to always be parsed as a generic, not a pair of comparisons – see this test program.

I suppose that if you did want two comparisons, you could add parentheses:

foo((b<c), d>(e))
    ^   ^

The approach Kotlin uses seems to require infinite lookahead. When encountering a <, the parser can't commit to whether to parse it as a comparison or as the start of a generic parameter list. For example, if the parser has seen this fragment of code so far:

foo(b<c,b<c,b<c,b<c,b<c,b<c,b<c, 

…these are all legal ways for the line to end:

1. foo(b<c,b<c,b<c,b<c,b<c,b<c,b<c, 4) (all comparisons)
2. foo(b<c,b<c,b<c,b<c,b<c,b<c,b<c, Int>>>>>>>()) (all generics)
3. foo(b<c,b<c,b<c,b<c,b<c,b<c,b<c, Int>>>()) (a mix of comparisons and generics!!)

(To play with this, look at lookahead.kt in the last link.)

That leaves me with a few questions:

• Are there peculiarities in Rust's grammar that would make adopting the same approach difficult or impossible?

• Was this approach ever considered in Rust's olden days, and rejected due to infinite lookahead or some other downside? (IMO, the benefit of killing turbofish would be well worth it.) Or did nobody realize it was a possibility?

• Adopting this in Rust would necessarily be a breaking change. Given (a<b, c>(d)), the 'default' parse must be as a generic rather than a comparison – because the comparison can be disambiguated by adding parentheses (((a<b), c>(d))) but there's nowhere to add parentheses that would make a generic the only valid interpretation. However, Rust currently does the opposite, defaulting to a comparison and requiring turbofish to force interpretation as a generic.

  Unfortunately, even if everyone suddenly agreed Rust should switch, AFAIK it would be way too late for the Rust 2018 edition. So… when's the next edition? :)

[BatmanAoD]

The advantage of ::< is that it looks obviously generics-related. This seems like a clear win for readability, especially compared to the disambiguate-with-parens strategy.

[BatmanAoD]

The advantage of ::< is that it looks obviously generics-related. This seems like a clear win for readability, especially compared to the disambiguate-with-parens strategy.

[Centril]

@comex

So… when's the next edition? :)

I'm betting on 2021 :)

[Centril]

@comex

So… when's the next edition? :)

I'm betting on 2021 :)

[artemshein]

For me :: looks obviously namespaces-related.

[artemshein]

For me :: looks obviously namespaces-related.

[BatmanAoD]

@artemshein I think that makes some sense, though. MyGeneric::<args...> is selecting the <args...> variant from the set of possible functions or types provided by MyGeneric. "Select x from the set X" is also how :: behaves when used on namespaces: X::x.

[BatmanAoD]

@artemshein I think that makes some sense, though. MyGeneric::<args...> is selecting the <args...> variant from the set of possible functions or types provided by MyGeneric. "Select x from the set X" is also how :: behaves when used on namespaces: X::x.

[artemshein]

@BatmanAoD haha, ok, maybe, but it's still ugly and doesn't reflect the generic parameters definition syntax (Option::<T>, anyone?) and introduces one more unnecessary thing to learn (big enough to have a full page of explanations like this https://matematikaadit.github.io/posts/rust-turbofish.html).

[artemshein]

@BatmanAoD haha, ok, maybe, but it's still ugly and doesn't reflect the generic parameters definition syntax (Option::<T>, anyone?) and introduces one more unnecessary thing to learn (big enough to have a full page of explanations like this https://matematikaadit.github.io/posts/rust-turbofish.html).

[BatmanAoD]

@artemshein The "ugliness" is of course a matter of personal aesthetic preference. Personally, coming from C++, when I saw the turbofish I immediately thought "oh, great, that's an improvement!" I realize I am in the minority in this view, though.

If consistency is the goal, I would personally rather see definition syntax match expression context, i.e. pub enum Option::<T>. I haven't proposed this because I assume that there are very few people who share that aesthetic preference and that the proposal would therefore immediately be shot down.

I also wouldn't call it an "unnecessary thing to learn." Clearly, per this discussion, it is necessary for Rust, even if it's different from C++!

[BatmanAoD]

@artemshein The "ugliness" is of course a matter of personal aesthetic preference. Personally, coming from C++, when I saw the turbofish I immediately thought "oh, great, that's an improvement!" I realize I am in the minority in this view, though.

If consistency is the goal, I would personally rather see definition syntax match expression context, i.e. pub enum Option::<T>. I haven't proposed this because I assume that there are very few people who share that aesthetic preference and that the proposal would therefore immediately be shot down.

I also wouldn't call it an "unnecessary thing to learn." Clearly, per this discussion, it is necessary for Rust, even if it's different from C++!

[Laaas]

Yeah I think just using turbofish everywhere might be superior. One idea I also had was since people dislike writing ::<>, change the :: part to !. Then it would be transmute!<f32>(x) and Option!<Vec!<u32>>. You could argue that it would be confused with macros, but is it really so wrong to consider generic items to be a macro of sorts, that generate a variant from the arguments? Though the change this would require might not make it worth it to switch away from the turbofish.

[Laaas]

Yeah I think just using turbofish everywhere might be superior. One idea I also had was since people dislike writing ::<>, change the :: part to !. Then it would be transmute!<f32>(x) and Option!<Vec!<u32>>. You could argue that it would be confused with macros, but is it really so wrong to consider generic items to be a macro of sorts, that generate a variant from the arguments? Though the change this would require might not make it worth it to switch away from the turbofish.

[BatmanAoD]

@Laaas I had the ! thought last night as well. I've seen it used with essentially the correct semantics (except with the reverse order) in non-programming contexts, e.g. referring to the version of Harry Potter from HPMOR as "Rational!Harry". So I think for some people, ! would be a natural way to express the concept.

[BatmanAoD]

@Laaas I had the ! thought last night as well. I've seen it used with essentially the correct semantics (except with the reverse order) in non-programming contexts, e.g. referring to the version of Harry Potter from HPMOR as "Rational!Harry". So I think for some people, ! would be a natural way to express the concept.

[petrochenkov]

chained comparisons are now banned in Rust

I assumed that they are banned now so they can be allowed in the future in their intuitive meaning - a < b < c == a < b && b < c.
In this case we cannot reuse the syntax for generic arguments.

backtracking is already present for some (uncommon) cases in the Rust parser

All these cases exist purely for diagnostics and can be removed without affecting Rust grammar.

This feature also directly interferes with const generic parameters.
When they are implemented, even the most innocent stuff like x < 10 && y > 10 will start looking like generic arguments.

[petrochenkov]

chained comparisons are now banned in Rust

I assumed that they are banned now so they can be allowed in the future in their intuitive meaning - a < b < c == a < b && b < c.
In this case we cannot reuse the syntax for generic arguments.

backtracking is already present for some (uncommon) cases in the Rust parser

All these cases exist purely for diagnostics and can be removed without affecting Rust grammar.

This feature also directly interferes with const generic parameters.
When they are implemented, even the most innocent stuff like x < 10 && y > 10 will start looking like generic arguments.

[varkor]

I assumed that they are banned now so they can be allowed in the future in their intuitive meaning - a < b < c == a < b && b < c.
In this case we cannot reuse the syntax for generic arguments.

That was one of the motivations, but note that chained comparisons in the same direction (e.g. a < b < c) are unambiguous with generics. The syntax that is not is a < b > c: I think a strong case could (and would) be made against this syntax for chained comparisons as it is far less readable than a < b && c < b (or equivalent). So I don't think this conflicts with generic arguments.

@petrochenkov:
Is this backtracking for diagnostic use only:
https://github.com/rust-lang/rust/blob/e9e7e53c31a3d1169ac81f0e82ca9168167ce711/src/libsyntax/parse/parser.rs#L3136-L3145
I couldn't tell from a quick look, but it looked plausibly like it could be used for non-error situations.

This feature also directly interferes with const generic parameters.
When they are implemented, even the most innocent stuff like x < 10 && y > 10 will start looking like generic arguments.

Expressions that are const generic arguments must be enclosed within a block, therefore no additional ambiguity over the tuple case given above is created.

[varkor]

I assumed that they are banned now so they can be allowed in the future in their intuitive meaning - a < b < c == a < b && b < c.
In this case we cannot reuse the syntax for generic arguments.

That was one of the motivations, but note that chained comparisons in the same direction (e.g. a < b < c) are unambiguous with generics. The syntax that is not is a < b > c: I think a strong case could (and would) be made against this syntax for chained comparisons as it is far less readable than a < b && c < b (or equivalent). So I don't think this conflicts with generic arguments.

@petrochenkov:
Is this backtracking for diagnostic use only:
https://github.com/rust-lang/rust/blob/e9e7e53c31a3d1169ac81f0e82ca9168167ce711/src/libsyntax/parse/parser.rs#L3136-L3145
I couldn't tell from a quick look, but it looked plausibly like it could be used for non-error situations.

This feature also directly interferes with const generic parameters.
When they are implemented, even the most innocent stuff like x < 10 && y > 10 will start looking like generic arguments.

Expressions that are const generic arguments must be enclosed within a block, therefore no additional ambiguity over the tuple case given above is created.

[petrochenkov]

Is this backtracking for diagnostic use only

Yes, if self.parse_ty_no_plus() (which is the only interpretation according to the grammar) fails, then we return an error on every path.

Expressions that are const generic arguments must be enclosed within a block, therefore no additional ambiguity over the tuple case given above is created.

Text and examples in that RFC contradict to itself, the precise rules were written out somewhere in the discussion thread, but were never moved into the RFC text properly.
The important part is that anything starting with a literal is certainly an expression and is unambiguously parsed as a const argument, so things like Type<1, 1 + 2> work without requiring braces.

[petrochenkov]

Is this backtracking for diagnostic use only

Yes, if self.parse_ty_no_plus() (which is the only interpretation according to the grammar) fails, then we return an error on every path.

Expressions that are const generic arguments must be enclosed within a block, therefore no additional ambiguity over the tuple case given above is created.

Text and examples in that RFC contradict to itself, the precise rules were written out somewhere in the discussion thread, but were never moved into the RFC text properly.
The important part is that anything starting with a literal is certainly an expression and is unambiguously parsed as a const argument, so things like Type<1, 1 + 2> work without requiring braces.

[comex]

Well, x < 10 && y > 10 wouldn’t be ambiguous because 10 can’t appear after a type or expression.

However, x < 10 && y > -10 would be...

…that is, under the parsing rule for const parameters that you describe. But it’s worth noting that const generics are not stable (or even fully implemented) yet, so it would be possible to change the rule back to what’s described in the RFC. It’s certainly nice to minimize the need for braces in const parameters, but on the other hand, “expressions that start with literals but reference variables later on” seems like a relatively narrow category.

[comex]

Well, x < 10 && y > 10 wouldn’t be ambiguous because 10 can’t appear after a type or expression.

However, x < 10 && y > -10 would be...

…that is, under the parsing rule for const parameters that you describe. But it’s worth noting that const generics are not stable (or even fully implemented) yet, so it would be possible to change the rule back to what’s described in the RFC. It’s certainly nice to minimize the need for braces in const parameters, but on the other hand, “expressions that start with literals but reference variables later on” seems like a relatively narrow category.

[comex]

...that said, if x<10 && y> - 10 (i.e. the const generics interpretation) would otherwise be valid syntax, and would thus conflict with x < 10 && y > -10, the grammar could and should still resolve the ambiguity in favor of the latter. That’s because in this case, both alternatives have a place to add parentheses to force them: you could force the first interpretation by writing x<{10 && y}> - 10, or force the second with (x < 10) && (y > -10). Thus, either option would be an acceptable default, in the sense of not making anything impossible to express.

Since it’s rather unlikely that 10 && y would typecheck (albeit not impossible), it would clearly be better to default to the other interpretation.

[comex]

...that said, if x<10 && y> - 10 (i.e. the const generics interpretation) would otherwise be valid syntax, and would thus conflict with x < 10 && y > -10, the grammar could and should still resolve the ambiguity in favor of the latter. That’s because in this case, both alternatives have a place to add parentheses to force them: you could force the first interpretation by writing x<{10 && y}> - 10, or force the second with (x < 10) && (y > -10). Thus, either option would be an acceptable default, in the sense of not making anything impossible to express.

Since it’s rather unlikely that 10 && y would typecheck (albeit not impossible), it would clearly be better to default to the other interpretation.

[estebank]

Is this backtracking for diagnostic use only:
https://github.com/rust-lang/rust/blob/e9e7e53c31a3d1169ac81f0e82ca9168167ce711/src/libsyntax/parse/parser.rs#L3136-L3145
I couldn't tell from a quick look, but it looked plausibly like it could be used for non-error situations

@varkor If you look at all 3 possible branches after that, the parser will always emit an error, either the original error or the specific error suggesting to use parenthesis around the expression. That code path could be removed and the compiler would start accepting that code as valid (which is what rust-lang/rust#42578 originally did), but the agreement was reached that it was a bad idea to change the grammar in that way.

I would love it if with the code you wrote in rust-lang/rust#53578, we could add the same kind of suggestions for people that may not have the full grasp of how to use the turbofish yet. We would not be fixing the ergonomics issue, but we'd be fixing part of the discoverability problem.

[estebank]

Is this backtracking for diagnostic use only:
https://github.com/rust-lang/rust/blob/e9e7e53c31a3d1169ac81f0e82ca9168167ce711/src/libsyntax/parse/parser.rs#L3136-L3145
I couldn't tell from a quick look, but it looked plausibly like it could be used for non-error situations

@varkor If you look at all 3 possible branches after that, the parser will always emit an error, either the original error or the specific error suggesting to use parenthesis around the expression. That code path could be removed and the compiler would start accepting that code as valid (which is what rust-lang/rust#42578 originally did), but the agreement was reached that it was a bad idea to change the grammar in that way.

I would love it if with the code you wrote in rust-lang/rust#53578, we could add the same kind of suggestions for people that may not have the full grasp of how to use the turbofish yet. We would not be fixing the ergonomics issue, but we'd be fixing part of the discoverability problem.