Add union math for intelligent indexing #6558
Conversation
Michael0x2a
force-pushed
the
add-union-math-for-intelligent-indexing
branch
from
57caca5
to
d198f38
Compare
Michael0x2a
force-pushed
the
add-union-math-for-intelligent-indexing
branch
from
d198f38
to
e134241
Compare
Fixes #6117 Fixes #5930 Currently both our plugin method hooks don't work with unions. This PR fixes this with three things: * Moves a bit of logic from `visit_call_expr_inner()` (which is a long method already) to `check_call_expr_with_callee_type()` (which is a short method). * Special-cases unions in `check_call_expr_with_callee_type()` (normal method calls) and `check_method_call_by_name()` (dunder/operator method calls). * Adds some clarifying comments and a docstring. The week point is interaction with binder, but IMO this is the best we can have for now. I left a comment mentioning that check for overlap should be consistent in two functions. In general, I don't like special-casing, but I spent several days thinking of other solutions, and it looks like special-casing unions in couple more places is the only reasonable way to fix unions-vs-plugins interactions. This PR may interfere with #6558 that fixes an "opposite" problem, hopefully they will work together unmodified, so that accessing union of literals on union of typed dicts works. Whatever PR lands second, should add a test for this.
|
Now that #6560 is merged, could you please add tests to check interactions between the two features? Then I will take a look at this. |
This pull request hacks on supports for union math when using Unions of
Literals to index into tuples, NamedTuples, and TypedDicts.
It also fixes a bug I apparently introduced. Currently, mypy correctly
reports an error with this code:
class Test(TypedDict):
foo: int
t: Test
# Error: int can't be assigned a str value
t.setdefault("foo", "unrelated value")
...but does not report an error with:
key: Literal["foo"]
t.setdefault(key, "unrelated value")
This diff should make mypy report an error in both cases.
Resolves python#6262.
Michael0x2a
force-pushed
the
add-union-math-for-intelligent-indexing
branch
from
e134241
to
be7210f
Compare
|
@ilevkivskyi -- ok, I added in a test testing a few basic interactions with TypedDicts. It didn't seem like this PR and #6560 really interfered with one another (as we hoped, I didn't have to make any code changes), so I stuck to just adding some basic sanity checks. LMK if you want something more in-depth. |
There was a problem hiding this comment.
Thanks! Looks good, I just have few suggestions for more docs and possible refactoring to reduce code (or rather logic) duplication.
Could you please also check this on internal code bases before merging? (I didn't do this with my unions PR and it caused some fallout.)
mypy/checkexpr.py
Outdated
| @@ -2607,40 +2617,59 @@ def nonliteral_tuple_index_helper(self, left_type: TupleType, index: Expression) | |||
| else: | |||
| return union | |||
|
|
|||
| def _get_value(self, index: Expression) -> Optional[int]: | |||
| def _get_values(self, index: Expression) -> Optional[List[int]]: | |||
There was a problem hiding this comment.
Please document this method. It is not new, but now it is sufficiently non-trivial to require a docstring with examples. Also make it non-private, essentially this is the same as try_getting_str_literals() but for integers instead of strings.
|
|
||
| return left_type.slice(begin, stride, end) | ||
| items = [] # type: List[Type] | ||
| for b, e, s in itertools.product(begin, end, stride): |
There was a problem hiding this comment.
The use of product would be clearer with a minimal example, like if there are two unions each with two elements (and stride is the same), the results would be a union with four elements.
| key_names.append(key_type.value) | ||
| else: | ||
| self.msg.typeddict_key_must_be_string_literal(td_type, index) | ||
| return AnyType(TypeOfAny.from_error) |
There was a problem hiding this comment.
Would it be practical to refactor this, so that this uses try_getting_str_literals()? It looks like there is some code duplication. (If yes, the function should be probably moved to nodes.py and imported both here and in the plugin.)
There was a problem hiding this comment.
Unfortunately, no -- this segment of code permits unicode keys but the try_getting_str_literals section doesn't.
IMO the correct thing to do is (a) modify try_getting_str_literals so it also allows unicode keys and (b) convert this entire function into a plugin.
But I think making this change fully correct is partly blocked on #6123, so I decided to defer the refactor for now.
|
@Michael0x2a The release branch will be cut tomorrow morning (SF time). Is this PR ready? |
|
@ilevkivskyi -- sorry, it took me a bit to work out some quirks with my setup so I could start testing this against our internal codebases. But I think (?) I've confirmed this PR introduces no new errors, so I went ahead and merged it in. |
This pull request hacks on supports for union math when using Unions of Literals to index into tuples, NamedTuples, and TypedDicts.
It also fixes a bug I apparently introduced. Currently, mypy correctly reports an error with this code:
...but does not report an error with:
This diff should make mypy report an error in both cases.
Resolves #6262.