Python 3 code generation

[gvanrossum]

We need a flag to generate Python 3 annotations instead of Python 2 style.

[impredicative]

Is a flag necessary? Can the tool do it based on the version of Python being used? Python 3.5+ users probably don't care about Python 2 style; certainly I don't. I can see the diff in other ways.

[gvanrossum]

The flag is not the important part. What we need to write is the code that inserts Python 3 style annotations, and that's not a trivial thing.

[gsemet]

+1 on the python 3.5+ annotation support. Would be so cool !

PS: you mean you mainly use python2 at dropbox?!!!

[kura]

So I guess this is the reason on Python 3.6 I get the following stack trace? Or do I need to create an issue for this?

  File "/home/kura/.virtualenvs/blackhole/bin/pyannotate", line 11, in <module>
    sys.exit(main())
  File "/home/kura/.virtualenvs/blackhole/lib/python3.6/site-packages/pyannotate_tools/annotations/__main__.py", line 45, in main
    generate_annotations_json(infile, tf.name)
  File "/home/kura/.virtualenvs/blackhole/lib/python3.6/site-packages/pyannotate_tools/annotations/main.py", line 37, in generate_annotations_json
    arg_types, return_type = infer_annotation(item.type_comments)
  File "/home/kura/.virtualenvs/blackhole/lib/python3.6/site-packages/pyannotate_tools/annotations/infer.py", line 38, in infer_annotation
    arg_types, return_type = parse_type_comment(comment)
  File "/home/kura/.virtualenvs/blackhole/lib/python3.6/site-packages/pyannotate_tools/annotations/parse.py", line 196, in parse_type_comment
    return Parser(comment).parse()
  File "/home/kura/.virtualenvs/blackhole/lib/python3.6/site-packages/pyannotate_tools/annotations/parse.py", line 205, in __init__
    self.tokens = tokenize(comment)
  File "/home/kura/.virtualenvs/blackhole/lib/python3.6/site-packages/pyannotate_tools/annotations/parse.py", line 188, in tokenize
    raise ParseError(original)
pyannotate_tools.annotations.parse.ParseError: Invalid type comment: (method-wrapper) -> bool

[gvanrossum]

@kura Can you file that as a separate bug? It's a simple fix, apparently there's an internal type whose name is method-wrapper. We need to figure out what to use instead (perhaps Callable?).

[kura]

@gvanrossum Will do!

[jabdoa2]

It will currently break existing annotation (for mypy):

Before:

    def get_express_config(self, value: str):
        """Parse short config version."""

After:

    def get_express_config(self, value: str):
        # type: (Union[int, str]) -> Dict[str, Any]
        """Parse short config version."""

Mypy dislikes this:

mypy mpf
mpf/config_players/coil_player.py:45: error: Function has duplicate type signatures

[gvanrossum]

Oh, that's a separate bug! It should not attempt to add an annotation if a Py3 annotation is already present.

[gvanrossum]

(Note: as a work-around for the original issue one could use https://github.com/ilevkivskyi/com2ann to post-process the Python 2 style annotations generated by PyAnnotate into Python 3 style annotations.)

[The-Compiler]

FWIW looks like Instagram released MonkeyType which can generate stub files or Python 3 annotations. I haven't tried it yet, though.

[gvanrossum]

I've tried it a little. Maybe we could write a script that converts our type_info.json into there sqlite3 (default) database format.

[omermazig-zz]

com2ann does not seems to work for this purpose. One of its tests specifically requires that it will ignore "#type : str".

I have not found a different way to process the Python 2 style annotations into Python 3 style annotations

[gvanrossum]

Perhaps you could write a script that translates the type_info.json file into the database format supported by MonkeyType, and use their backend?