Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Class level scoping is miscompiled #862

Open
JelleZijlstra opened this issue Jun 5, 2021 · 3 comments
Open

Class level scoping is miscompiled #862

JelleZijlstra opened this issue Jun 5, 2021 · 3 comments

Comments

@JelleZijlstra
Copy link
Collaborator

Compiling the below:

from typing import Final

class StringParser:
    DEFAULT_TOKEN: Final = -1
    d: Final = {DEFAULT_TOKEN: 1}

And running it produces:

$ mypyc fud.py
$ python -c 'import fud'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "fud.py", line 5, in <module>
    d: Final = {DEFAULT_TOKEN: 1}
KeyError: 'DEFAULT_TOKEN'

The relevant snippet of C code is:

    cpy_r_r39 = CPyStatic_globals;
    cpy_r_r40 = CPyStatics[3]; /* 'DEFAULT_TOKEN' */
    cpy_r_r41 = CPyDict_GetItem(cpy_r_r39, cpy_r_r40);

This looks like it's looking up the name in the globals dict instead of the class dict.

It doesn't reproduce with a positive integer, float, or string Final.

This is minimized from code in Black, originally found by @ichard26.

@msullivan
Copy link
Collaborator

Oh, argh.

I think the core issue here is that we don't really understand class-level scoping.
... and it comes up with negative integers because mypyc doesn't understand negative integers are constants.

@JukkaL
Copy link
Collaborator

JukkaL commented Jun 8, 2021

There seem to be two things to fix:

  • Support negative integers as constants (or more generally, various integer operations involving constants expressions, such as addition and bitwise ops).
  • Fix class-level scoping in general.

@JukkaL
Copy link
Collaborator

JukkaL commented Dec 11, 2022

python/mypy#14283 fixes the negative integer case.

JukkaL added a commit to python/mypy that referenced this issue Dec 15, 2022
Now mypy can figure out the values of final variables even if the
initializer has some operations on constant values:
```
A: Final = 2  # This has always worked
A: Final = -(1 << 2)  # This is now supported
B: Final = 'x' + 'y'  # This also now works
```
Currently we support integer arithmetic and bitwise operations, and
string concatenation.

This can be useful with literal types, but my main goal was to improve
constant folding in mypyc. In particular, this helps constant folding
with native ints in cases like these:
```
FLAG1: Final = 1 << 4
FLAG2: Final = 1 << 5

def f() -> i64:
    return FLAG1 | FLAG2  # Can now be constant folded
```

We still have another constant folding pass in mypyc, since it does some
things more aggressively (e.g. it constant folds some member expression
references).

Work on mypyc/mypyc#772.

Also helps with mypyc/mypyc#862.
@hauntsaninja hauntsaninja changed the title Negative integer class constant is miscompiled Class level scoping is miscompiled Feb 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants