Fix decimal

[srittau]

Fix various problems with decimal and add missing annotations.

[srittau]

mypy crash reported as python/mypy#5347.

[srittau]

Crash is now fixed in mypy master.

[JelleZijlstra]

Thanks! I restarted Travis, will take another look after it passes.

[JelleZijlstra]

Thanks! Left a few comments.

[JelleZijlstra]

This conditional definition makes it look confusingly like int is not supported in Python 3. In fact, it is redundant in Python 2 too, because it's automatically promoted to float. It would be clearer to have int either in both or in neither of the branches.

[JelleZijlstra]

I think this should actually be omitted because the effect of having it is that the type checker will think setting any attribute is allowed. However, the implementation actually only allows the attributes that are known to exist on the class.

[srittau]

I replaced it with a comment saying so, for future reference.

[JelleZijlstra]

Most of these return bool, although I haven't checked all of those. I'm guessing divmod returns a Tuple[Decimal, Decimal].

[srittau]

Good catch. I am rechecking the return values of all Context methods.

[srittau]

I am not sure why the travis build does not start automatically after the latest patches.

[JelleZijlstra]

I closed and reopened, and now Travis seems to be running.

[gvanrossum]

I wonder if this broke something. I've got code that's essentially (though not quite)

float(Decimal() if bool() else 0)

(Though that example doesn't provoke it.)

The if/else expression's type is revealed as object, and
float(object()) is invalid. Maybe it was valid before? Or maybe the
join previously returned Union[Decimal, int] and now deteriorated to
object?

I kind of suspect #2323, since
it removed the SupportsFloat and SupportsInt base classes from Decimal
(the last typeshed sync before this afternoon's was
python/mypy#5373, which synced typeshed 3
commits before 2323).

[srittau]

I think it's reasonable to assume that this change triggered the problem, although I suspect a bug in mypy. Since SupportsAbs and SupportsFloat are protocols, they should not be needed as base classes.

The following test cast demonstrates the problem:

from typing import SupportsFloat


class Floaty:
    def __float__(self): ...


class Supporter(SupportsFloat):
    def __float__(self): ...


reveal_type(Floaty() if bool() else 0)
reveal_type(Supporter() if bool() else 0)

The former is revealed as builtins.object, the latter as typing.SupportsFloat. As short term solution would be to add the base classes to Decimal again, but I believe this should be fixed in mypy.

[JukkaL]

@srittau Mypy works like by design, though it's clearly suboptimal in this case. It's not clear how this should be fixed in mypy. It's easy to imagine how this could work in simple cases, but a general solution seems difficult to define -- so that it's both practical and has the desired theoretical properties. Luckily the problems seem to arise pretty infrequently, and they can often be worked around by adding an explicit protocol base class. Can we add the base classes back? We should probably add a comment explaining why they are there so that they won't be removed again in the future.

If you want to propose a mypy fix, I'm happy to discuss it. I've looked into this a bit but gave up as it didn't seem worth the effort at the time.

[srittau]

The problem I have with that is that it basically makes protocols useless, because the if .... else ... construct will not work with any protocol. This would mean we'd have to add all protocols to all classes again.

[JukkaL]

This is the first time I've seen this cause problems, so it's a bit unfair to say that it makes protocols basically useless. I proposed a potential mypy improvement in python/mypy#5392 (comment), which could partially address the issue.

[srittau]

Well, useless in the sense that whenever someone comes up with a case where a protocol is used in an if ... else ... construct, we'd have to add the protocol as base class to the involved classes or add them preemptively. I suspect that it didn't come up much so far is because protocols are not used much yet in typeshed.

But just implementing the suggestion from typing/issues#564 would likely make this problem more pronounced.