python import separates decorator from function

[tbnorth]

Although only when the decorator spans multiple lines:

@base_task(
    targets=['img/who_map.png', 'img/who_map.pdf'],
    file_dep=[data_path('phyto')],
    task_dep=['load_data'],
)
def make_map():
    """make_map - plot the Thompson / Bartsh / WHO map
    """
...

Leo 5.4, build 20170105133315, Thu Jan  5 13:33:15 EST 2017
Git repo info: branch = master, commit = 5bd7034831a1
Python 2.7.5, PyQt version 4.8.5
Windows 7 AMD64 (build 6.1.7601) SP1

[edreamleo]

A new unit test shows that the old importers have the same behavior. Moreover, this bug does not cause any perfect import tests to fail.

[edreamleo]

Yikes. I never dreamed decorators could be so complex. As the example shows, decorators may contain nested parens. Furthermore, multi-line decorators could have lines ending in comments, and those comments could also contain parens.

As a result, there is absolutely no way that regex patterns can discover the end of multi-line decorators. And even if they could, they would be horribly complex. See this discussion.

In short, the code that moves decorators must do a full rescan of multi-line decorators. This calls into question the entire strategy of handling decorators separately from lines that start defs and classes.

Maybe not :-) We can probably assume that everything between the start of a multi-line decorator (easy regex) and the next class/def lines is, in fact, part of the decorator. This "localized" solution is likely to be far safer than messing with the crucial scanning code in python_i.gen_lines.

The alternative would be to recognize decorators as actually starting defs and classes. This will complicate the "starts block" logic, but it may well be the simplest solution.

[edreamleo]

In the end, I did have to modify python_i.gen_lines, but the changes were straightforward. You could say that the new code is even a bit simpler than the old. However, the new python_i.starts_decorator method could not be done in any reasonable regex, and may have to be changed a bit.

Update: ef5ccf4 should put this issue to bed.