Improve Docstring location information

[analog-cbarber]

The Docstring init method currently does this:

        self.value: str = inspect.cleandoc(value.rstrip())
        self.lineno: int | None = lineno
        self.endlineno: int | None = endlineno

The net result leading whitespace is removed from each line. This is what we want,
but it now means that the there is no information about how much to add to get the
correct column offset.

It would be nice if the class could provide enough information to be able to associate
characters in the docstring with their line/column in the original source. Accordingly,
I suggest:

1. add a col_offset field indicating the size of the margin that was removed

2. (bonus feature) add a method to convert an offset into the cleaned up doc string into original line,col.
   Something like:

   def offset_to_line_col(self, offset:int) -> Tuple[int,int]:
       line = self.lineno + self.value.count('\n', 0, offset)
       col = self.col_offset + offset - self.value.rfind('\n', 0, offset) - 1
       return line, col

Implementing this would either require pulling implementation of cleandoc into the module to
expose the margin or comparing the cleaned up string with the original to infer the correct value.

[analog-cbarber]

You can only infer the column offset for the line after the first based on this information alone. To get the
offset of the first line, it would have to be passed in (from the AST node).

[analog-cbarber]

And of course there are tabs. I wonder how many python projects actually allow tabs?

[pawamoy]

Since we store the original source code in the lines collection, we could use the docstring's lineno and endlineno to fetch the original docstring, including the first line. I'm not sure what the use-case is here though.

Would that work for you if we add a property to docstrings that returns the original value? Something like this:

@property
def original(self) -> str:
    return "\n".join(self.parent.lines_collection[self.parent.filepath][self.lineno-1:self.endlineno])

[analog-cbarber]

I think that might work. The use case is the ability to produce error messages that correctly report the line and column of tokens in the doc string so that IDEs can navigate directly to the write characters and/or highlight errors directly in the source.

[pawamoy]

I see, thanks, that's what I thought I remembered from our discussions about relative cross-references.

Lets do this then, I'll add this property to the Docstring class 🙂 PRs welcome too!