New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect errors for fstrings + generators #3385

Closed
ns-cweber opened this Issue May 19, 2017 · 3 comments

Comments

Projects
None yet
2 participants
@ns-cweber

ns-cweber commented May 19, 2017

mypy version: 0.511
python version: 3.6.1

Given this valid program:

from typing import Iterator

def stringify(it: Iterator[int]) -> Iterator[str]:
    return (f"{x}" for x in it)

def stringify2(it: Iterator[int]) -> Iterator[str]:
    for x in it:
        yield f"{x}"

def stringify3(it: Iterator[int]) -> Iterator[str]:
    return (str(x) for x in it)

def stringify4(it: Iterator[int]) -> Iterator[str]:
    for x in it:
        yield str(x)

I get errors for the first two variations:

foo.py:4: error: Generator has incompatible item type "int"
foo.py:8: error: Incompatible types in yield (actual type "int", expected type "str")

It seems that mypy doesn't recognize that fstrings are produce a str. Interestingly, the yield error didn't show up in my old mypy version 0.501.

@ns-cweber

This comment has been minimized.

Show comment
Hide comment
@ns-cweber

ns-cweber May 19, 2017

Strangely, this is fine:

def stringify5(it: Iterator[Tuple[int, int]]) -> Iterator[str]:
    return (f"({x}, {y})" for x, y in it)

... but this is not:

class Point:
    def __init__(self, x, y) -> None:
        self.x = x
        self.y = y

    def __str__(self) -> str:
        return f"({self.x}, {self.y})"

def stringify6(it: Iterator[Point]) -> Iterator[str]:
    return (f"{pt}" for pt in it)

So it seems like it's the unpacking syntax (... for x, y in ...) that makes the difference, and not whether or not the iterator type is primitive.

ns-cweber commented May 19, 2017

Strangely, this is fine:

def stringify5(it: Iterator[Tuple[int, int]]) -> Iterator[str]:
    return (f"({x}, {y})" for x, y in it)

... but this is not:

class Point:
    def __init__(self, x, y) -> None:
        self.x = x
        self.y = y

    def __str__(self) -> str:
        return f"({self.x}, {self.y})"

def stringify6(it: Iterator[Point]) -> Iterator[str]:
    return (f"{pt}" for pt in it)

So it seems like it's the unpacking syntax (... for x, y in ...) that makes the difference, and not whether or not the iterator type is primitive.

@ns-cweber ns-cweber changed the title from Incorrect errors for new fstrings to Incorrect errors for fstrings + generators May 19, 2017

@ns-cweber

This comment has been minimized.

Show comment
Hide comment
@ns-cweber

ns-cweber May 19, 2017

It looks like it doesn't have anything to do with the unpacking; it seems to pertain to the handling of ast.FormattedValue. Mypy handles this correctly:

def stringify(it: Iterator[int]) -> Iterator[str]:
    return (f"asdf {x}" for x in it)

According to GreenTreeSnakes' FormattedValue docs:

If the string contains a single formatting field and nothing else the node can be isolated otherwise it appears in JoinedStr.

In this most recent example, using an fstring with tokens besides the formatted value produces a JoinedStr token. Maybe mypy handles JoinedStr correctly but not FormattedValue?

ns-cweber commented May 19, 2017

It looks like it doesn't have anything to do with the unpacking; it seems to pertain to the handling of ast.FormattedValue. Mypy handles this correctly:

def stringify(it: Iterator[int]) -> Iterator[str]:
    return (f"asdf {x}" for x in it)

According to GreenTreeSnakes' FormattedValue docs:

If the string contains a single formatting field and nothing else the node can be isolated otherwise it appears in JoinedStr.

In this most recent example, using an fstring with tokens besides the formatted value produces a JoinedStr token. Maybe mypy handles JoinedStr correctly but not FormattedValue?

@JelleZijlstra

This comment has been minimized.

Show comment
Hide comment
@JelleZijlstra

JelleZijlstra May 19, 2017

Collaborator

Yes, it looks like the bug is that mypy interprets f"{x}" as having the type of x.

Collaborator

JelleZijlstra commented May 19, 2017

Yes, it looks like the bug is that mypy interprets f"{x}" as having the type of x.

lincolnq added a commit to lincolnq/mypy that referenced this issue May 21, 2017

Parse each format-string component separately
Fixes python#3385.

The old code evaluated each FormattedValue as an expression potentially
returning the type of the value, rather than a string. But that's wrong,
because a FormattedValue can exist on its own when it's not being joined
to any other format strings. The new code evaluates each FormattedValue
by synthesizing '{}'.format(expr) and then, if necessary, joins them in
JoinedStr using ''.join(items).

lincolnq added a commit to lincolnq/mypy that referenced this issue May 27, 2017

Parse each format-string component separately
Fixes python#3385.

The old code evaluated each FormattedValue as an expression potentially
returning the type of the value, rather than a string. But that's wrong,
because a FormattedValue can exist on its own when it's not being joined
to any other format strings. The new code evaluates each FormattedValue
by synthesizing '{}'.format(expr) and then, if necessary, joins them in
JoinedStr using ''.join(items).

gvanrossum added a commit that referenced this issue May 30, 2017

Parse each format-string component separately (#3390)
Fixes #3385.

The old code evaluated each FormattedValue as an expression potentially
returning the type of the value, rather than a string. But that's wrong,
because a FormattedValue can exist on its own when it's not being joined
to any other format strings. The new code evaluates each FormattedValue
by synthesizing '{}'.format(expr) and then, if necessary, joins them in
JoinedStr using ''.join(items).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment