Do more with parameter inspection #296

mthuurne · 2020-11-27T04:14:43Z

Fixes #217

Known issues: (to be fixed after release)

Warn on duplicate fields #229: we don't warn about duplicate type or keyword fields yet: type is not easy to implement, keyword is barely used
Format integer literals like the source #294: doesn't happen all that often
Lookup of name in annotation fails on reparented object #295: annoying but not easy to fix

codecov · 2020-11-27T04:16:10Z

Codecov Report

Merging #296 (aa9e6f1) into master (a06aee5) will increase coverage by 0.19%.
The diff coverage is 87.17%.

@@            Coverage Diff             @@
##           master     #296      +/-   ##
==========================================
+ Coverage   84.39%   84.58%   +0.19%     
==========================================
  Files          23       23              
  Lines        3735     3782      +47     
  Branches      766      783      +17     
==========================================
+ Hits         3152     3199      +47     
  Misses        386      386              
  Partials      197      197

Impacted Files	Coverage Δ
pydoctor/epydoc2stan.py	`87.36% <85.91%> (+1.19%)`	⬆️
pydoctor/model.py	`88.86% <100.00%> (+0.13%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a06aee5...aa9e6f1. Read the comment docs.

These could be typos or outdated documentation. In either case, it's worth alerting the user to this, so the documentation can be fixed.

For the reader of the API docs, knowing the type of a parameter is a lot better than knowing nothing at all.

Instead of explicitly setting "Undocumented" as the body in specific cases, we just leave the body attribute as None and on presentation we substitute "Undocumented" if necessary. This removes an inconsistency where 'param' fields did have the "Undocumented" string while 'return' fields did not.

Most functions with None as their return type don't actually return a value. In cases where they do, the user can use a 'returns' or 'rtype' field to force the "Returns" entry to appear.

The information we extract from type annotations can be useful even if there is no docstring.

mthuurne · 2020-11-27T13:05:47Z

pydoctor/epydoc2stan.py

+    def handle_keyword(self, field: Field) -> None:
+        name = self._handle_param_name(field)
+        if name is not None:
+            # TODO: How should this be matched to the type annotation?


There are two functions in Twisted that use @keyword, but neither of them have type annotations at the moment, so I think it's fine to resolve this TODO later.

Do we want to support @keyword ... can't we just use @param ? :)

I see that in the previous code keyword was the same as param... so maybe deprecated keyword and just use @param everywhere.

The annotation would be useful only if all keywords have the same type: if for example one keyword is a str and another keyword is an int, the annotation will be object or Any or Union[int, str], which isn't very helpful to the end user. So the author will have to use type fields to document the types of the individual keywords.

Do we want to support @keyword ... can't we just use @param ? :)

I see that in the previous code keyword was the same as param... so maybe deprecated keyword and just use @param everywhere.

The purpose of @keyword is to document keywords that can be captured by the **kwargs parameter. I think it's useful to have a separate field for those, since otherwise we wouldn't be able to warn about @param being used with a non-existing parameter name. That warning led to several documentation fixes in Twisted.

We already had this warning for the 'param' field, but documenting a type for a parameter that does not exist is equally worthy of receiving a warning. This also solves an issue where a 'type' field for a non-existing parameter that precedes the 'param' field hides the warning for that 'param' field, since having an entry in 'self.types' is how we decide that a parameter exists.

This is a partial implementation of twisted#229: ideally we'd want to warn on duplicate `type` and `keyword` fields as well, but for `type` this is currently not easy to implement and `keyword` isn't used enough to make it a priority.

adiroiban

It looks good. Only minor comments.

It would have been nice to have the sample API doc enabled so that we could see in HTML in a browser all the cased for there is a new test

For now, I have used https://pydoctor--296.org.readthedocs.build/en/296/api/.

adiroiban · 2020-11-27T17:10:39Z

pydoctor/epydoc2stan.py

+    def handle_keyword(self, field: Field) -> None:
+        name = self._handle_param_name(field)
+        if name is not None:
+            # TODO: How should this be matched to the type annotation?


Do we want to support @keyword ... can't we just use @param ? :)

I see that in the previous code keyword was the same as param... so maybe deprecated keyword and just use @param everywhere.

adiroiban · 2020-11-27T17:19:10Z

pydoctor/test/test_epydoc2stan.py

+def test_func_keyword(capsys: CapSys) -> None:
+    """Test handling of @keyword."""


Suggested change

def test_func_keyword(capsys: CapSys) -> None:

"""Test handling of @keyword."""

def test_func_parameter_as_keyword(capsys: CapSys) -> None:

"""A warning is raised when a parameter is documented as a @keyword."""

adiroiban · 2020-11-27T17:21:09Z

pydoctor/test/test_epydoc2stan.py

+        )
+
+def test_func_arg_not_inherited(capsys: CapSys) -> None:
+    """Do not warn about non-existing parameters from inherited docstrings."""


Suggested change

"""Do not warn about non-existing parameters from inherited docstrings."""

"""No warning is emitted when a subclass is implementing an inherited function with different parameters."""

adiroiban · 2020-11-27T17:23:23Z

pydoctor/test/test_epydoc2stan.py

@@ -128,6 +159,83 @@ def f(a):
    classic_fmt = docstring2html(classic_mod.contents['f'])
    assert annotation_fmt == classic_fmt

+def test_func_param_duplicate(capsys: CapSys) -> None:
+    """Warn when the same parameter is documented more than once."""


~~Do we already have a test when the same parameter has multiple @type definitions?~~ this is #229

adiroiban · 2020-11-27T17:24:39Z

pydoctor/test/test_epydoc2stan.py

+
+def test_func_undocumented_return_something() -> None:
+    """When the returned value is undocumented and its type is not None,
+    include a the "Returns" entry in the output.


I don't understand this docstring....

So if the return type is not None and is not documented, it is added as an undocumented span.

I guess that it handles this case https://pydoctor--296.org.readthedocs.build/en/296/api/pydoctor.astbuilder.ASTBuilder.html#parseFile

Can we also include <tr class="fieldStart"><td class="fieldName">Returns</td><td colspan="2"> in the expected output?

Yes, that is the case that it handles. I'll add a bit more detail to the docstring, in the hope that it will be easier to understand.

I added both <td> tags in the expected output, that indeed makes the test easier to read. However, I left out the <tr>, since that is not really part of what is being tested here and adding more output increases the chance that the test will require maintenance on unrelated changes.

mthuurne added 6 commits November 27, 2020 13:37

Warn about documented parameters that don't exist in the definition

ed33eb9

These could be typos or outdated documentation. In either case, it's worth alerting the user to this, so the documentation can be fixed.

Warn if @keyword is used to document an ordinary parameter

99d442b

Output type information from annotations of undocumented parameters

bebc528

For the reader of the API docs, knowing the type of a parameter is a lot better than knowing nothing at all.

Do not include a "Returns" entry just for a None annotation

fd5f642

Most functions with None as their return type don't actually return a value. In cases where they do, the user can use a 'returns' or 'rtype' field to force the "Returns" entry to appear.

Format parameters also for functions without a docstring

c8aa8c9

The information we extract from type annotations can be useful even if there is no docstring.

mthuurne commented Nov 27, 2020

View reviewed changes

mthuurne force-pushed the more-param-inspection branch from a0cd674 to dc6ca1d Compare November 27, 2020 16:41

mthuurne added 2 commits November 27, 2020 17:47

Warn when a parameter is documented more than once

b9d4e05

This is a partial implementation of twisted#229: ideally we'd want to warn on duplicate `type` and `keyword` fields as well, but for `type` this is currently not easy to implement and `keyword` isn't used enough to make it a priority.

mthuurne force-pushed the more-param-inspection branch from dc6ca1d to b9d4e05 Compare November 27, 2020 16:48

mthuurne marked this pull request as ready for review November 27, 2020 16:54

mthuurne requested a review from adiroiban November 27, 2020 16:54

Add test case for fetching constructor parameters from an empty class

5c1bd74

adiroiban approved these changes Nov 27, 2020

View reviewed changes

mthuurne mentioned this pull request Nov 27, 2020

Warn on duplicate fields #229

Open

mthuurne added 2 commits November 27, 2020 20:24

Make test docstrings more clear

3bc6e94

Expand the expected HTML in test_func_undocumented_return_something

aa9e6f1

mthuurne merged commit 429b7a0 into twisted:master Nov 27, 2020

mthuurne deleted the more-param-inspection branch November 27, 2020 19:40

mthuurne mentioned this pull request Dec 2, 2020

DOC: Fix RST Consolidated Fields reference #308

Merged

mthuurne mentioned this pull request Dec 11, 2020

some invalid epytext markup is not reported as such #7

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Do more with parameter inspection #296

Do more with parameter inspection #296

mthuurne commented Nov 27, 2020 •

edited

Loading

codecov bot commented Nov 27, 2020 •

edited

Loading

mthuurne Nov 27, 2020

adiroiban Nov 27, 2020

mthuurne Nov 27, 2020

mthuurne Nov 27, 2020

adiroiban left a comment

adiroiban Nov 27, 2020

adiroiban Nov 27, 2020

adiroiban Nov 27, 2020

adiroiban Nov 27, 2020 •

edited

Loading

adiroiban Nov 27, 2020

mthuurne Nov 27, 2020

		def test_func_keyword(capsys: CapSys) -> None:
		"""Test handling of @keyword."""

	"""Do not warn about non-existing parameters from inherited docstrings."""
	"""No warning is emitted when a subclass is implementing an inherited function with different parameters."""

Do more with parameter inspection #296

Do more with parameter inspection #296

Conversation

mthuurne commented Nov 27, 2020 • edited Loading

codecov bot commented Nov 27, 2020 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adiroiban left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adiroiban Nov 27, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mthuurne commented Nov 27, 2020 •

edited

Loading

codecov bot commented Nov 27, 2020 •

edited

Loading

adiroiban Nov 27, 2020 •

edited

Loading