Python member operation does not work on lists because of overloaded ndarray equality

[mitar]

One more surprising results:

cache = []
cache.append(numpy.ndarray([1,2,3]))
cache.append(numpy.ndarray([4,5,6]))
numpy.ndarray([4,5,6]) in cache

Bam:

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

This prevents the use of general caching code which stores values in the cache. (List is used here to support non-hashable values and because this example is extracted from a stack-based algorithm.)

[eric-wieser]

How do you propose we fix this?

Overloading == to be element-wise is far too useful for masking, such as x[x==10] = 0.

And if we want to keep that behavior, then there is no way to fix this.

List is used here to support non-hashable values

Caching non-hashable (mutable) values is a bad idea, because the value can change after you cache them.

[mitar]

Is x[x==10] = 0 really so much cleaner than x[x.eq(10)] = 0 or something like that? I was really surprised when I learned about this numpy behavior. To me == and other operators really operate on top-level values, not at elements. You do not have [1,2,3] == 3 return True.

Django has a slightly alternative approach with their query constructor language. They have something called Q object, we could have E instead (for element). Then you could do:

x[E(x)==10] = 0

Slightly more verbose, but very clear that you are comparing elements of x against 10.

[eric-wieser]

really so much cleaner than x[x.eq(10)] = 0 or something like that?

When you also want to include <,<=, >=, >, and !=, then yes, spelling out the operators is a lot worse.

The django-style approach is an interesting one, but it would be dangerously easy to write a[a == 10] instead of a[E(a) == 10], where the former couldn't error because a[True] is well-defined

[mitar]

Hm, if we fix ==, then a[a == 10] would be always equal to a[False]. So at least it fails closed.

But this does not hold true for !=. Anyway, a[False] might have to be in that case be deprecated as well, if we want to catch such error.

[njsmith]

Changing the definition of == to return a scalar would break literally millions of people's programs. I appreciate the enthusiasm, but there's no way we can do that.