Improved (in)equality testing with non-boolean results

[nickovs]

This is a version of #5479 which makes use of a new flag in the flags word of mp_obj_type_t to indicate if various shortcuts can not be used when performing equality and inequality tests, as per the comment in the discussion of that PR.

The semantics of the new TYPE_FLAG_NO_EQUALITY_SHORTCUTS are that when this is clear (or uninitialised, in a static structure) the code can assume that this class (a) only ever returns a boolean result, (b) is reflexive, (c) only implements the __eq__ operator and not the __ne__ operator and (d) can not be equal to an instance of any different class that also has this flag clear. If the flag is set then at least one of these assumptions does not hold.

Currently four built-in classes have the flag set: float and complex are non-reflexive (since nan != nan) while bytearray and frozenszet instances can equal other builtin class instances (bytes and set respectively).

The flag is set for any new class defined by the user. In the future, code could be added to analyse user-defined classes to see if either of the __eq__ or __ne__ special methods are defined and clear the flag if both are absent in the class and all its superclasses.

Adding the flag (and setting it for a few built-in classes) looks like it reduced the amount of code and it should now take the fast paths for most common cases, while further improving the compatibility with CPython behaviour.

The PR also expands the test cases to cover some cases that previously did not function as CPython did.

[dpgeorge]

Thanks for following up!

Initial tests for size difference gives:

   bare-arm:  +108 +0.164% 
minimal x86:  +220 +0.149% 
   unix x64:  +112 +0.022% 
unix nanbox:   +32 +0.007% 
      stm32:   +92 +0.024% PYBV10
     cc3200:  +120 +0.065% 
    esp8266:   +64 +0.009% GENERIC
      esp32:    -8 -0.001% GENERIC[incl -32(data)]
        nrf:   +76 +0.052% pca10040
       samd:   +96 +0.095% ADAFRUIT_ITSYBITSY_M4_EXPRESS

That's very reasonable.

[nickovs]

@dpgeorge Something strange is happening with the test coverage. The new tests that I just added do cover the one line in objtype.c that shows as uncovered when I run make coverage_test on my Linux machine. I don't see how the tests could complete without exercising this path and I don't see how that the compiler could ever optimise this away. If you get the chance to run the coverage tests perhaps you could look to see if line 1398 in objtype.c is being hit.

[dpgeorge]

The new tests that I just added do cover the one line in objtype.c that shows as uncovered when I run make coverage_test on my Linux machine.

Yeah, that happens sometimes. It seems that gcov is not always accurate with the lines it hits. I'll double check it though.

[dpgeorge]

I see that the check and warning for comparison between str and bytes is now gone. There is a test for this in tests/basics/bytes_compare3.py but, because enabling the warning is a compile-time option, the test will actually pass even if the warning is not printed.

IMO the check and warning should be retained. I don't think there's an elegant way to add it, I think there needs to be explicit tests in mp_obj_equal_bop() for it (similar to how it worked before).

[nickovs]

I've reinstated the warning for comparisons of strings and bytes.

[dpgeorge]

Is this correct? I think it should be || not &&... anyway, I can take it from here, I think I can make this a bit simpler.

[nickovs]

There's definitely something wrong either here or the line below. Let me investigate and fix before you merge.

[dpgeorge]

I've already fixed it and have done a few other clean ups.... I can push it as a new PR for further review.

[dpgeorge]

See #5593 for a slightly reworked version of this PR.

[nickovs]

I only just saw your last comment after I pushed a fix with some other improvements. I'm happy to go with your version if you prefer.

[dpgeorge]

I only just saw your last comment after I pushed a fix with some other improvements.

No worries. I'll see if your version of the string handling is smaller and work it in.

[dpgeorge]

This doesn't work for object representations C&D where floats are not objects (they are inline values stored in the mp_obj_t itself), so it fails the nan==nan test (see nanbox failure on Travis).

[dpgeorge]

Ok, I've merged #5593 without the final commit from here. See commits c3450ef through c96a2f6

Thanks very much @nickovs for the hard work getting this done, it's tough making such deep changes to the core!

[nickovs]

Sounds good.

Thanks very much @dpgeorge for the hard work of creating MicroPython in the first place!

[mcdeoliveira]

Happy to see this done! Great job @nickovs and @dpgeorge.