Infix syntax specification for functions/operators

[quinnj]

This is a follow up to a post on julia-dev about providing the ability to check containment similar to Python, Ruby, SQL, etc.

'4 in [1,2,3,4,5]' -> true

This is extremely intuitive syntax and nonconflicting with our current use of 'in' while still being conceptually similar.

And while I'm on Santa's lap and on the topic of syntax readability, can I request we use 'and' and 'or' as aliases for short circuit operators && and ||, a la python? I think they are keywordy enough that people avoid using them for variable names while doing much for clear, readable syntax.

Thoughts?

Update: Changed title to reflect the desire for generalized infix syntax

[diegozea]

+1 to have in as an infix operator for containment checking.

I like the idea of and and or but I'm not very sure. || and && are not difficult to read... And way is and && and not & ?

[stevengj]

Please no and and or synonyms. Multiple syntaxes for the same keyword/operator are not a good direction to go.

We already have has(X, element) for sets and dictionaries; that's quite readable without adding new infix operators.

[JeffBezanson]

Yes, I prefer to steal as few names as possible, and on top of that having synonyms in the language is a style nightmare.

Our operators are already just function names with special syntax, so in principle this could be extended to more names. But special syntax for the in/has function strikes me as too much of a special case. There is also a bit of confusion as to whether x in dict would check for x as a key, or as a (key,val) tuple which is what the dict iterates as elements. So you might need 2 special cases.

[quinnj]

@stevengj, I understand your point on multiple syntaxes, though I think it's more of a problem when it involves operators/functions that are very similar but slightly different (think '=' and '->' in R). Where I originally proposed and and or to be straight aliases for && and ||, I think it would be less of a problem. But in general I see the point of trying to steal as few names as possible and think that's fair.

With in on the other hand, my original intent was for it to be infix for contains() rather than has(), which I don't completely understand the overlap between those 2 functions, but I at least know that:

julia> has([1,2,3,4,],3)
ERROR: no method has(Array{Int32,1},Int32)
julia> contains([1,2,3,4],3)
true
julia> contains(["one"=>1,"two"=>2,"three"=>3,"four"=>4],("three",3))
true

With the latter two being the primary case. I think that solves the Dict problem because we would support dicts how contains support dicts, as a regular iterable that would only work on a tuple (key,val).

[quinnj]

I think I see what @JeffBezanson was saying now though about potential confusion.

julia> contains(["one"=>1,"two"=>2,"three"=>3,"four"=>4],3)
false
julia> contains(["one"=>1,"two"=>2,"three"=>3,"four"=>4],"three")
false

Which means if someone tried to check 2 in ["one"=>1,"two"=>2] and got false, that would be confusing. But it seems like this is a contains() problem. Should there be some kind of check that the input-type match how the collection iterates?

[toivoh]

See also
https://groups.google.com/d/msg/julia-dev/c1TK_1m_4F0/WCWjrDCyws8J
https://groups.google.com/d/msg/julia-dev/6E1q5C72d7M/Hx9x-CbXEBsJ
https://groups.google.com/d/msg/julia-dev/QCVDrcdxUMs/pec8ZuQt-8AJ
I personally think that infix in helps readability a lot. I wouldn't mind a general infix function call syntax either, like (x in y) (that has the benefit of specifying precedence explicitly). I don't see how in would be more or less ambiguous than has or contains; I think we'll just have to dedide on a convention on what it/they mean, and stick with it.

[StefanKarpinski]

I don't buy the ambiguity argument – we just need to decide what in means – the syntax issue is completely orthogonal. I would be ok with in having special syntax, but I would also be in favor of some more general form of generalized infix syntax like what @toivoh is suggesting. That would be very handy for a variety of things, beyond just item in collection.

[JeffBezanson]

If we address #2347 , it makes sense to write k in keys(dict) which is nice.

[StefanKarpinski]

Yes, that would be very nice.

[toivoh]

I like that.

[blakejohnson]

Essentially implemented in b6da170.

Now there is the new question of whether it makes sense to have a shorthand for asking if something is not in a collection. To me, v not in S is a lot easier to read than !in(v, S) or !(v in S), but elsewhere Julia uses ! for boolean negation.

[stevengj]

v not in S seems very un-Julian, since we don't allow not anywhere else (you can't say x not == y, nor would we want to). I would prefer to stick with !in(v, S) or !(v in S). (Unlike Python, which has a general not operator instead of !.)

[StefanKarpinski]

This is what you get for adding syntax :-[. Sorry, I think we're sticking with !(v in S) which reads just fine to me. I do still like having v in S and it might be nice to allow that style more generally.

[blakejohnson]

I'll get used to !(v in S), eventually. I think this issue can now be closed.

[stevengj]

@JeffBezanson, shouldn't contains be deprecated now?

[blakejohnson]

@stevengj contains is the name of a new method acting on Strings.

[stevengj]

Why not in for strings?

[blakejohnson]

See #4153 and #4184. I guess the idea was that in represents whether something is a member of an iterable collection, whereas contains would be for substructure containment. So,

julia> "ab" in "abc"
false
julia> "ab" in ["ab", "bc", "cd"]
true
julia> contains("abc", "ab")
true

That said, I think "ab" in "abc" => false is confusing, and we should make the first example above equivalent to the last.

[kmsquire]

After 143395e :

julia> "ab" in "abc"
ERROR: use contains(x,y) for string containment
 in in at string.jl:479

[StefanKarpinski]

Fully agree that "ab" in "abc" => false is confusing – it should be an error instead (which it now is).

[stevengj]

What is the rationale for making it an error rather than simply returning true?

[JeffBezanson]

Because subsequence testing and set containment are not the same operation. Using the element-of operator for finding substrings is an abuse of notation I don't believe I've ever seen.

[kmsquire]

Python does it.

[JeffBezanson]

If every math and theory textbook is one day updated to use python's definition of set containment, I will concede the point.

[kmsquire]

No argument, just pointing out where the idea came from.

[stevengj]

We aren't talking about the set-containment operator ∈, we are talking about the English word "in".

[JeffBezanson]

My question is, why can't there be a function that simply checks whether a value is one of those generated by something iterable? This seems like a totally obvious primitive function to me. What should this function be called?

[StefanKarpinski]

We're not talking about the English word "in" we're talking about the Julia operator in and what it should mean. We need to be far, far less ambiguous and overloaded than English.

[kmsquire]

In #4184 , Stefan gave a nice reponse when I espoused a different point of view about allowing multiple semantic meanings for the same function. Of course, you'll never get away entirely from having multiple meanings for the same function, and you need to balance this against needless function proliferation, but I can respect the view that in languages with multiple dispatch like Julia, it makes life easier down the road if semantic overloading is minimized.

In this case, if in is extended to substring containment, does that also extend to sub array containment? They're pretty similar. But if so, what do you do with arrays of arrays, which can both be elements and subarrays of that containing array? You could just disallow this for arrays (not a common operation), of course, and this isn't an issue with strings, but then it becomes a special case, and the general preference seems to minimize special cases as well.

Anyway, I'm happy enough with in meaning "is element of", and using something different for "is substring of".

[JeffBezanson]

I agree it can be ok to take an "informal" approach to the semantics of some functions; it's just that this one is so primitive. It's such an extreme case that you almost don't want it to be a generic function, and just iterate and check isequal, except that would be too slow in many cases. This is also a reason why I'm ok with having syntax for it; anywhere you can say for x in y, x in y now also works and refers to the same meanings of everything.

[BobPortmann]

"anywhere you can say for x in y, x in y now also works and refers to the same meanings of everything"

To me this is the best argument for the present syntax (and to not have string containment included in in).

[StefanKarpinski]

Yes, it's a very compelling argument.