Add .end_lineno attribute to pyclbr _Objects

[kebab-mai-haddi]

BPO	38307
Nosy	@terryjreedy, @stevendaprano, @brandtbucher, @kebab-mai-haddi, @kebab-mai-haddi
PRs	bpo-38307:completes the Stack implementation to yield ending line for each class. #16466 bpo-38307: Add end_lineno attribute to pyclbr _Objects #24348

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2022-03-19.08:04:21.182>
created_at = <Date 2019-09-28.17:26:30.267>
labels = ['type-feature', 'library', '3.10']
title = 'Add .end_lineno attribute to pyclbr _Objects'
updated_at = <Date 2022-03-19.08:04:21.182>
user = 'https://github.com/kebab-mai-haddi'

bugs.python.org fields:

activity = <Date 2022-03-19.08:04:21.182>
actor = 'iritkatriel'
assignee = 'none'
closed = True
closed_date = <Date 2022-03-19.08:04:21.182>
closer = 'iritkatriel'
components = ['Library (Lib)']
creation = <Date 2019-09-28.17:26:30.267>
creator = 'aviral'
dependencies = []
files = []
hgrepos = []
issue_num = 38307
keywords = ['patch']
message_count = 8.0
messages = ['353467', '353468', '385657', '385807', '385814', '385816', '385819', '386085']
nosy_count = 5.0
nosy_names = ['terry.reedy', 'steven.daprano', 'brandtbucher', 'aviral', 'kebab-mai-haddi']
pr_nums = ['16466', '24348']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue38307'
versions = ['Python 3.10']

[kebab-mai-haddi]

Currently, the readmodule returns the starting line of the classes but not the end line. This should be provided as well in order to get the scope of the class, mainly to help check what all imports were used in a class.

[stevendaprano]

What readmodule are you referring to?

3.6 is in feature-freeze. The earliest that any enhancement could be added is version 3.9.

Can you explain how having an ending line number as well as the starting line number can be used to check which imports are used in a class?

[kebab-mai-haddi]

How do I generate the endline no? Initially, I could
do, stack[-1][0].end_lineno = start[0] - 1 but how do I this now given that the recent changes are operating on the AST instead of the token stream?

[kebab-mai-haddi]

I have made the changes and tested, my builds were successful. Please review it?

CC @brandtbucher, @steven.daprano

[terryjreedy]

The answer, Aviral discovered, is that current ast nodes have end_lineno attributes, so the patch amounts to consistently copying them.

I don't understand Aviral's use case either, but an item on my IDLE wish list is to be able to move class and function definitions within a file by moving the Class and Function entries within the browser tree. This requieres end lines and having it be an attribute is easier and more accurate than recalculating it. Merely clicking on an entry could highlight the whole definition. Other people might think of other uses.

An issue in the patch is inserting parameter 'end_line' after 'line'. Normally, new parameters have to go at the end, but:

1. Inserting 'end_line' after 'line' makes the code easier to read.

2. The parameter or parameters at the end have default values. If we added 'end_line=None', we would have to consider what to do when it is omitted.

3. https://docs.python.org/3/library/pyclbr.html#module-pyclbr, which I rewrote in 2017, documents readline and readline_ex as the public call interface. They return a hierarchical structure that includes Functions and Classes in the dict values. Their signatures are intentionally omitted and the attributes are only listed after saying "Users are not expected to create instances of these classes."

I posted "What is the pyclbr public API" to pydev asking about this issue.

[kebab-mai-haddi]

Hey Terry, thanks for commenting. I have a few questions to ask you, please pardon my lack of awareness.

This requieres end lines and having it be an attribute is easier and more accurate than recalculating it.

How do you recalculate the end_lineno?

Since all the objects that start, have an end too, would end_lineno=None make any sense?

I could not make anything out of your third point where you mention readline and readline_ex. Can you explain that point?

I posted "What is the pyclbr public API" to pydev asking about this issue.

Can you share the link of your post?

As for my use case, I need the scope of a class and a function in the new tool that I am developing. My tool is to generate a new type of UML diagrams for a given codebase. Do check it out and leave your critical feedback: https://github.com/kebab-mai-haddi/gruml

I need to know about the scope (start and end) so that I can deduce whether the "used_at" attribute of an object is within a class. This will help me deduce whether a particular class (or a function) uses any imported object. This is to deduce the dependency of a class on other classes and functions.

Currently, I have to edit the pyclbr and make it custom, then, had to Dockerize the whole thing. So, want to contribute to cpython so that this feature is present at the source itself.

[terryjreedy]

If IDLE were to recalculate the end from start, it would have to more or less parse forward to find the end of the statement. This is why I considered it a speculative wish.

would end_lineno=None make any sense?
Only as a kludge if a default were required because of the signature position.

Point 3: The existing doc implies but does not exactly state that the _Object signature are private, so that we could break any possibly existing (but possibly not, and discouraged) use. I think back-compatibility is discussed in the devguide.

Posts do not have a link until distributed, and then you can find it on mail.python.org. The title is sufficient to do that.

[terryjreedy]

New changeset 000cde5 by Aviral Srivastava in branch 'master':
bpo-38307: Add end_lineno attribute to pyclbr Objects (GH-24348)
000cde5