New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve stub files generated by stubgenc for pybind11 modules #5850

Merged
merged 9 commits into from Nov 22, 2018

Conversation

Projects
None yet
2 participants
@wiktorn
Copy link
Contributor

wiktorn commented Oct 28, 2018

Following changes fixes stubgenc that generates invalid *pyi files in following cases:

  1. Class constructors may sometimes get Any type as return value. After this change - for init None is the default
  2. For modules using pybind11, base class is pybind11_object which is non-existent (see #5814). After this change pybind11_object is removed
  3. Stubgenc doesn't add any import statements other than referencing some typing.* objects. If module references other types in different modules/packages no import statement is added. This pull requests adds import statements to referenced types

I'm not sure about 2, maybe pybind11 library should provide dummy pybind11_object in its module and annotate types, so dummy object will be assumed as superclass.

wiktorn added some commits Oct 28, 2018

Handle fully qualified types
Strip module name from type information if its the same as annotated module or
add import statement.
@gvanrossum

This comment has been minimized.

Copy link
Member

gvanrossum commented Oct 28, 2018

@gvanrossum
Copy link
Member

gvanrossum left a comment

Sorry for the late review -- this got caught in a busy period. I have a few requests for changes, but in general I think this is fine. (I do wonder if maybe we should switch to a more robust approach to handling argument lists rather than repeated splitting and stripping. But that's for a follow-up PR.)

Show resolved Hide resolved mypy/stubgenc.py Outdated

if sig:
sig_types = []
for arg in (arg.split(':', 1) for arg in sig.split(',')):

This comment has been minimized.

@gvanrossum

gvanrossum Nov 19, 2018

Member

I'm sorry, but this code is inscrutable, and I have no easy example in the stdlib to even test it (no C function in the stdlib has type annotations in its docstring). At the very least please don't use the same name (arg) for the two different variables -- the inner arg takes on string values like x: int or foo: C while the outer arg would take on list values like ['x', ' int'] and ['foo', ' C']. You might also want to add some comments explaining the different cases (len(arg) being 1 or not).

Show resolved Hide resolved mypy/stubgenc.py Outdated
Show resolved Hide resolved mypy/stubgenc.py Outdated
Show resolved Hide resolved mypy/stubgenc.py Outdated
Show resolved Hide resolved mypy/stubgenc.py Outdated
Show resolved Hide resolved mypy/test/teststubgen.py
Show resolved Hide resolved mypy/test/teststubgen.py
@gvanrossum
Copy link
Member

gvanrossum left a comment

(Sorry, marking as "changes requested".)

@wiktorn

This comment has been minimized.

Copy link
Contributor

wiktorn commented Nov 21, 2018

I hope that I correctly identified changes that you've requested. The only thing that may require further discussion is handling of pybind11_object.

@gvanrossum
Copy link
Member

gvanrossum left a comment

Everything looks good except I'd still like to see some improvements on the "inscrutable" for-loop. :-)

Thinking it through I believe the special case for pybind11_object is acceptable -- I assume you don't have a say in what pybind11 generates anyway...

if len(arg) == 1:
sig_types.append(arg[0].strip())
# convert signature in form of "self: TestClass, arg0: str" to list [[self, TestClass], [arg0, str]]
for arg_type in (arg.split(':', 1) for arg in sig.split(',')):

This comment has been minimized.

@gvanrossum

gvanrossum Nov 21, 2018

Member

Please rewrite this as follows:

# Convert signature in form of "self: testmod.TestClass, arg0: str" to list ["self: testmod.TestClass", "arg0: str"]
# and update imports with relevant modules (e.g. "testmod").
for arg in sig.split(','):
    arg_type = arg.split(':', 1)
    if len(arg_type) == 1:
    # etc.

wiktorn and others added some commits Nov 22, 2018

Review fixes.
* Fix PEP8 line length
* simplify logic
* introduce testcase for generate_c_function_stub that verifies docstring parsing
@gvanrossum

This comment has been minimized.

Copy link
Member

gvanrossum commented Nov 22, 2018

LGTM, waiting for tests (I fixed one typo).

@gvanrossum gvanrossum merged commit 93c6e2c into python:master Nov 22, 2018

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@gvanrossum

This comment has been minimized.

Copy link
Member

gvanrossum commented Nov 22, 2018

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment