Ad-hoc operator overloading

[SiegeLord]

Rendered.

This RFC is in many ways an alternative to #392.

[Gankra]

How do multi-method operators like slice work?

[SiegeLord]

How do multi-method operators like slice work?

Each method gets its own #[operator] attribute.

[sfackler]

How will I as a consumer of a library be able to easily find out if I can use + or whatever with a type defined in that library?

[Gankra]

Can slice be partially implemented, then? Only implement [..n] or whatever?

[SiegeLord]

How will I as a consumer of a library be able to easily find out if I can use + or whatever with a type defined in that library?

Typically you will still implement the traits from core::ops, so that your type can be used in most generic code (same reason why'd you implement Clone rather than make your own duplication method). Additionally, I imagine we could enhance rustdoc to indicate what attributes are attached to the method.

Can slice be partially implemented, then? Only implement [..n] or whatever?

Yeah.

[ftxqxd]

I like this idea. Currently I think it is bad that we describe operators using traits: people will try to use those operator traits as trait bounds, which in my opinion is a bad idea as the operator could mean anything: for numbers it’s addition, but for strings it’s concatenation. Using + for concatenation is not inherently bad, but is an example of how one shouldn’t assume that + represents addition (and thus is commutative, transitive, and so on). I think we should instead still have an Add trait in the standard library, with a #[operator="add"] decoration, but say that Add represents numerical addition, and anything else wanting to implement + in a non-number-like way would implement it as an inherent method, or perhaps as an impl of another concatenation/whatever trait (with an #[operator="add"] decoration on the relevant method).

I also like the way that slicing could potentially be split up with this. Strings really shouldn’t implement [a..b], [a..], and [..b], because they would probably take byte indices, which aren’t normally what you want. However, the [] syntax is very useful. It would be great if we could split Slice into AsSlice and SliceRange or something.

I’d also like to make the suggestion that the #[operator] use the actual operator symbol instead of just some arbitrary name, i.e., use #[operator="+"] and #[operator="[a..]"] instead of #[operator="add"] and #[operator="slice_from"]. That would mean that we could extend this to arbitrary operators in the future (albeit probably with some extra complications).

[blaenk]

I agree with @P1start's final point about making it use the actual operator, to at least leave open the possibility of arbitrary operators.

[netvl]

How do you write things like generic ranges in this case? Currently range() is defined like this:

pub fn range<A: Add<A, A> + PartialOrd + Clone + One>(start: A, stop: A) -> Range<A>

There is a nice and clean bound which only requests what is absolutely needed for the range. How will this function look with ad-hoc operator overloading? The reason why it works in C++, for example, because templates there are just raw substitutions. But I don't think we want to introduce C++-like templates into Rust.

BTW, Haskell is not a valid example of ad-hoc overloading. You can't overload (+) function without implementing a type class. If you do write your own (+) for the type you need without implementing Num type class, this new definition will take over (or there will be a conflict with imported name, I don't remember) the original one, so you won't be able to add numbers, for example. There is no function overloading in any form in Haskell.

[bill-myers]

Thinking about this, it makes a lot of sense.

After all, when you write "x.foo(y)", the compiler doesn't lookup a Foo trait and then call its foo method, but rather it just performs method lookup for foo (and obviously this is a good thing).

There doesn't seem to be any reason for "x + y" to work differently.

Traits with operators are still possible and would work.

[bill-myers]

Why not just use a C++-ish syntax like this:

fn op +(&self, rhs: &uint) -> uint;

or a Scala-ish notation like this:

fn +(&self, rhs: &uint) -> uint;

or even this daring syntax (not sure if this is unambiguous and parsable):

fn (&self + rhs: &uint) -> uint;

One could also imagine allowing any sequence of non-[A-Za-z0-9] characters (with exceptions to avoid conflicts) as an operator.

[SiegeLord]

It's a possibility. I went with a more conservative choice for implementation ease.

[brendanzab]

Without committing to supporting this RFC, the Haskell style fn (+)(lhs: T, rhs: U) -> V might make more sense, seeing as binary operators are called in a different way to regular functions.

[reem]

This RFC would be more convincing if it included examples of useful patterns this would allow that can't be created using today's system. Personally I'm not really aware of many cases, but I could be easily convinced if I saw code that was much nicer to write or work with under this RFC.

[huonw]

How do you write things like generic ranges in this case? Currently range() is defined like this:

@netvl in some ways this is a bug; range could/should be using a Range trait with succ and pred like Haskell's Enum type class. This allows it to be used with general enumerated types, like char.

[huonw]

Currently I think it is bad that we describe operators using traits: people will try to use those operator traits as trait bounds, which in my opinion is a bad idea as the operator could mean anything: for numbers it’s addition, but for strings it’s concatenation. Using + for concatenation is not inherently bad, but is an example of how one shouldn’t assume that + represents addition (and thus is commutative, transitive, and so on).

We could say that impls of Add should be a commutative, associative operation (and remove the broken ones in the stdlib). It's perfectly reasonable to define a trait to have specific meaning that's not strictly enforced by the compiler, as long as it doesn't lead to memory unsafety without using unsafe. In fact, capturing shared behaviour like this is basically the point of using traits. (That is, an "invalid" impl of Add would just lead to confusion and annoyance at the library author.)

(This is not a comment about this RFC specifically, just pointing out that that reasoning doesn't necessarily apply.)

[ftxqxd]

We could say that impls of Add should be a commutative, associative operation (and remove the broken ones in the stdlib). It's perfectly reasonable to define a trait to have specific meaning that's not strictly enforced by the compiler, as long as it doesn't lead to memory unsafety without using unsafe.

Yes, but then you don’t get the nice a + b syntax for non-addition-like things. My point was that at the moment, Add is impld as a way of doing operator overloading (which just happens to be done through a trait), but it’s treated in bounds as a normal trait (with the associated not-enforced-by-the-compiler invariants). I agree that that traits should be allowed to have properties that aren’t enforced by the compiler, and in fact that was part of my point: we should have a way of having an Add trait with the related invariants (commutativity, transitivity) while at the same time having the ability to use the quite unrelated syntactic sugar of a + b without having to obey those invariants (and thus not implementing the Add trait specifically).

Basically, I’m trying to say that we are ‘abusing’ traits by making Add (and most other operator traits) represent two different things: the syntax a + b, which is purely sugar and should be unrelated to its actual functionality, and the idea of addition, which is what traits should actually be used for.

[huonw]

Why would you want to use + for something that's not addition-like? That seems like an abuse of operator overloading, i.e. a canonical example of what proponents of no-overloaded-operators complain about (yes, I don't like our current impl of Add for String). It's very disingenuous to say that the a + b syntax is unrelated to the concept of addition, since + is the symbol for addition.

[japaric]

This RFC would be more convincing if it included examples of useful patterns this would allow that can't be created using today's system. Personally I'm not really aware of many cases, but I could be easily convinced if I saw code that was much nicer to write or work with under this RFC.

This would make the indexing and slicing operators actually useful without having to wait for HKT, the current method signature of index and slice are too limiting and pretty much only work for Vec/[T] and String/str. With this proposal you could define:

• Matrix indexing that returns rows:

// Self = Mat<T>
fn index<'a>(&'a self, row: &uint) -> Row<'a, T> { .. }
let row = mat[0];
let elem = mat[1][2];

• Matrix slicing that returns a sub matrix view

// Self = Mat<T>
fn slice_or_fail<'a>(&'a self, start: (uint, uint), end: (uint, uint)) -> View<'a, T> { .. }
assert_eq!(mat[(1, 2) .. (3, 4)].size(), (2, 2));

• Slicing of strided slices, etc

None of that is possible with the current Index/Slice traits

---

About the proposal, I got two questions:

• Will it be possible to overload operations on types not defined in the current crate? Example

// crate foo
trait CharIndex {
    #[operator="[]"]
    fn index(&self, pos: uint) -> char;
}

impl CharIndex for String {}

• What happens in this case:

impl Foo {
    #[operator="[]"]
    fn index(&self, index: uint) -> Bar { .. }
}

trait MyIndex {
    #[operator="[]"]
    fn index(&self, index: uint) -> Baz;
}

// is it possible to overload the operator again?
impl MyIndex for Foo { .. }

foo[0]  // what's the return value: Bar or Baz?

---

Other thoughts:

• This RFC could prevent the scenario of getting the stdlib cluttered with HKT and non-HKT variants of traits used for operator overloading.
• This RFC would allow unsafe (no bounds checked) indexing on *const [T]: unsafe { raw_slice[0] } (I think @gankro wanted this)
• This RFC would probably remove the need of having two very similar indexing traits: Index and IndexGet.

[SiegeLord]

I’d also like to make the suggestion that the #[operator] use the actual operator symbol instead of just some arbitrary name, i.e., use #[operator="+"] and #[operator="[a..]"] instead of #[operator="add"] and #[operator="slice_from"]. That would mean that we could extend this to arbitrary operators in the future (albeit probably with some extra complications).

Yeah, that might be a good idea. My only concern is the arbitrariness of the a in [a..]. We'd have to spec what exactly can go in there and what it would mean, if anything.

BTW, Haskell is not a valid example of ad-hoc overloading.

Yes, I'm aware this is not practical to take advantage of in Haskell. Nevertheless, if you want to have a hard time (i.e. only use your non-Num operators in a module that doesn't import Num's functions), the possibility is there. That's sort of a general issue with Haskell, and not specific to operator overloading.

@japaric in all cases those would act like normal methods. E.g. you will be able to add new operators to types outside the crate (you'll have to use your own trait or a wrapper type).

In the second case, it'd be a conflict that you'll have to two options of resolving:

• Using UFCS and the raw method names (you lose syntax sugar in this case)
• Selectively don't import the trait or use trait bounds (today's workabout for the lack of UFCS). This does allow you to keep using the syntax sugar.

[Gankra]

Re usecases: some people would also like to be able to unsafely index or +/- a ptr.

This RFC makes me feel some feels. I think this is the sort of solution that makes the most sense long term. I'm not sure if it would be appropriate for 1.0, though. Lots of work to get this going, I imagine.

There will need to be some checks for arity involved (I don't recall seeing this in the proposal?). If actual symbols are used, then something will need to be done to handle arity-overloaded operators. "-" being the most obvious one. For those interested in generalizing this to a future with "any operator", how would this be handled? Would all operators be candidates for unary and binary forms? Or perhaps we could have a syntax like "_ - _", "- _", "[_ ...]", etc. to distinguish.

It might also make sense to put some sanity constraints on the ownership of the LHS. For instance, += with &LHS makes basically no sense in decent code, but this proposal seems to admit it as possible.

[Ericson2314]

I think also allowing #[operator="...] on normal functions would be more consistent. And overloading is probably a real pain for type checking, so I think the choices should be non-overloaded #[operator="...] (pretty close to the Haskell way), or the current system.

[arielb1]

@Ericson2314

This shouldn't cause new type-checking problems – a <OP> b will just become a.<OP>(b) but with different autoref rules.

Anyway, it will be interesting to know how this handles Index vs. IndexMut.

[Ericson2314]

Ah ok, the overloading and methods-only restriction go together. I am not sure how I feel about exploiting identifier ambiguities in this way however---the system is analogous to an unhygienic macro. But yes, it does get the job done.

[ftxqxd]

@huonw

Why would you want to use + for something that's not addition-like? That seems like an abuse of operator overloading, i.e. a canonical example of what proponents of no-overloaded-operators complain about (yes, I don't like our current impl of Add for String). It's very disingenuous to say that the a + b syntax is unrelated to the concept of addition, since + is the symbol for addition.

+ represents addition in the context of mathematics, but in programming that’s not necessarily the case. Languages like Python use + for string concatenation; it’s quite useful and it’s obvious what it does. It’s not great being forced to sacrifice nice syntax simply because it doesn’t match the precise mathematical definition of the symbol. Yes, adding other operators is a better solution (for this particular case) in my opinion, but until Rust gets that, using + for concatenation can do. Python and similar languages had the option to make a new operator for concatenation, but didn’t as they saw it unnecessary.

And + isn’t the only example of an operator that can be overloaded nicely without preserving their primary meaning. |, &, and ! can be useful in various scenarios, ranging from types representing patterns or PEGs to DSL-like constructs to generate a mini language. It seems strange to me to forcefully couple the syntax with one particular meaning, instead of allowing a broader scope of meanings, with common sense determining what is an appropriate use. Operators are basically the same as method names, and I see no reason, for example, to force all methods named add to represent mathematical addition: adding items to a set could use the same name. The reason that this seems fine is because add can have multiple meanings that depend on the context. I see no reason why we can’t accept that + can have multiple meanings, mathematical or not, in the same way that names can have multiple meanings.

[pcwalton]

I am sympathetic to this but I think that Haskell is not a valid comparison, because being able to overload + is not something you can do on a type-based basis in the same scope. To me the strongest argument is that it's similar to how dot notation works today: a.add(b) works with multiple signatures of add. This is something that some Haskellers don't like :) But it is how Rust works, and extending to operators makes sense to me for consistency's sake (and we do something similar with Deref/Index and friends right now anyhow).

I believe this can be added backwards compatibly post 1.0, so I'll mark this as postponed.

[brendanzab]

Languages like Python use + for string concatenation;

As a counterexample, Haskell and Scala use (++) for both string and vector concatenation, and D uses (~).

[ghost]

I'm glad to see something like this being proposed. The current approach, while conceptually simple, just does not seem to scale very well at all.

Basically, I’m trying to say that we are ‘abusing’ traits by making Add (and most other operator traits) represent two different things: the syntax a + b, which is purely sugar and should be unrelated to its actual functionality, and the idea of addition, which is what traits should actually be used for.

I agree with this 100%. I have a semigroup library where having an operator like * makes sense for each impl, but the operation does not necessarily represent arithmetic multiplication. Due to the fact that operators are tied to traits, and given coherence restrictions, I have to go through various contortions to make it work.

[huonw]

@darinmorrison, that's a perfectly reasonable multiplication, in particular, it is associative; I'm not concerned about arithmetic operations mainly mathematical convention. I'm afraid you'll need to be more specific about what the contortions are because that doesn't seem outrageous to me.

[ghost]

I'm afraid you'll need to be more specific about what the contortions are because that doesn't seem outrageous to me.

Sorry, I should have been more specific. In particular, it's a nuisance to have to wrap everything in S in order to get * to work reasonably across the various impls. In an earlier version I made Semigroup a trivial extension of Mul but this just doesn't work because you can't provide an impl of Mul for things like Option. Worse, I can't define a new operator (as far as I know), so the solution is to wrap everything in a trivial struct S which creates an unnecessary layer everywhere which you can see here.

I suppose there are ways to work around this using macros (see here) but that seems a bit unsatisfying, not to mention complicated.

[ghost]

@darinmorrison, that's a perfectly reasonable multiplication, in particular, it is associative; I'm not concerned about arithmetic operations mainly mathematical convention.

I'm not sure mathematical convention is a good measure because it varies too much depending on your perspective. For instance, * would still make sense as an operator for Magma. In that case, we have nothing other than the fact that there is an operation. Indeed, we might choose to start there instead and define Semigroup as AssociativeMagma, etc.

[brendanzab]

@darinmorrison Have you seen my num-rs library? It shows one way to do it using the current operator traits.

[ghost]

@bjz I hadn't seen that. Looks very nice! I'll have to take a closer look, thanks.

[ghost]

@bjz Ah, having looked at it closer now, I think I see what you mean. This is actually how I was doing Semigroup originally (using std::ops::Mul<_,_>). Like I mentioned earlier though, this is a problem if you intend to provide impls of the algebraic traits for non-numeric types, which is my primary interest here. It seems that a choice is forced between either providing a wrapper for all builtin types and providing std::ops::Mul<_,_> impls for each wrapper or just doing it once like with S. The latter seemed to be less noisy and simpler but still not ideal.