ast.unparse produces: 'FunctionDef' object has no attribute 'lineno' for valid module

[TheRobotCarlson]

BPO	46073
Nosy	@isidentical, @sobolevn, @TheRobotCarlson
Superseder	bpo-44896: AttributeError in ast.unparse
Files	test.py: test file showing the ast failure

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2022-03-19.15:01:30.742>
created_at = <Date 2021-12-14.16:38:50.128>
labels = ['3.10', 'library', '3.9', 'type-crash', '3.11']
title = "ast.unparse produces: 'FunctionDef' object has no attribute 'lineno' for valid module"
updated_at = <Date 2022-03-19.15:01:30.742>
user = 'https://github.com/TheRobotCarlson'

bugs.python.org fields:

activity = <Date 2022-03-19.15:01:30.742>
actor = 'iritkatriel'
assignee = 'none'
closed = True
closed_date = <Date 2022-03-19.15:01:30.742>
closer = 'iritkatriel'
components = ['Library (Lib)']
creation = <Date 2021-12-14.16:38:50.128>
creator = 'TheRobotCarlson'
dependencies = []
files = ['50490']
hgrepos = []
issue_num = 46073
keywords = []
message_count = 7.0
messages = ['408546', '408561', '408563', '408565', '408566', '408567', '408574']
nosy_count = 3.0
nosy_names = ['BTaskaya', 'sobolevn', 'TheRobotCarlson']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '44896'
type = 'crash'
url = 'https://bugs.python.org/issue46073'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

[TheRobotCarlson]

Test file linked. When unparsing the output from ast.parse on a simple class, unparse throws an error: 'FunctionDef' object has no attribute 'lineno' for a valid class and valid AST. It fails when programmatically building the module AST as well.

It seems to be from this function:

cpython/Lib/ast.py

Line 790 in 1cbb887

def get_type_comment(self, node):

[sobolevn]

Looks like this little patch can solve this problem:

   def unparse(ast_obj):
       unparser = _Unparser()
---    return unparser.visit(ast_obj)
+++    return unparser.visit(fix_missing_locations(ast_obj))

But, I am not sure we should call this helper here.

Another approach is to refactor get_type_comment function into something like this:

    def get_type_comment(self, node):
        comment = None
        if hasattr(node, 'lineno'):
            comment = self._type_ignores.get(node.lineno)
        comment = comment or node.type_comment
        if comment is not None:
            return f" # type: {comment}"

I feel like this is more suitable for this case. Even ast.pyi has this line:

    # TODO: Not all nodes have all of the following attributes
    lineno: int
    col_offset: int

So, I am going to propose a PR with the second option. Please, send your feedback!

[sobolevn]

Moreover, there's always an option to pass lineno and column explicitly:

from ast import *

# if we build the module manually and try it directly
value = Module(
    body=[
        FunctionDef(
            name="items_needed",
            args=arguments(
                posonlyargs=[],
                args=[],
                kwonlyargs=[],
                kw_defaults=[],
                defaults=[],
            ),
            body=[Return(value=Constant(value="test"))],
            decorator_list=[],
            lineno=1,
            column=1,
        )
    ],
    type_ignores=[],
)

unparse(value)  # ok

[TheRobotCarlson]

The second solution seems more optimal, in my opinion. I monkey patched the function like this in my own code:

def get_type_comment(self, node):
    comment = self._type_ignores.get(node.lineno) if hasattr(node, "lineno") else node.type_comment
    if comment is not None:
        return f" # type: {comment}"

Thanks!

[TheRobotCarlson]

I don't think passing lineno and column is preferred. It makes code generation harder because lineno and column are hard to know ahead of when code is being unparsed.