Add plugin API to specify fine grained dependencies

[ilevkivskyi]

Currently plugins can add type information that requires reprocessing if another node is updated. This requires additional fine grained dependencies to be correct. For example:

# file a.py
@dataclass
class A:
    x: int

# file b.py
from a import A
@dataclass
class B(A):
    y: int

B(1, 2)

Updating type of a.A.x requires re-checking (more precisely re-semantically analyzing) the B definition, and the B call later in b.py.

The idea is to give a possibility for plugins to add dependencies. It looks like the simplest way is to add an add_dependencies method to semantic analyzer interface with a signature like def (trigger_fullname: str, node_to_recheck: Node) -> None. Then hooks for class decorators, base classes, and metaclasses can use it when modifying a TypeInfo.

[msullivan]

Interestingly, solving this particular case doesn't seem to require adding any dependencies at all:
the module already has dependencies from the attributes, so it seems like the important bit is mostly to make sure that aststrip properly strips the methods added.

This might not be true for all plausible extensions, though?

[msullivan]

I've got a proof of concept that solves this case at least. I'm going to spend some time investigating just how bad the memory use penalty would be for making copies of trees so we can ditch aststrip.

Update: Making copies of all the ASTs before semantic analysis gives a 20% increase in daemon memory usage without a cache on S, which I think is on the border of palatability.

[ilevkivskyi]

TBH, I already forgot the details about this test case. It may be you are right. I just checked my local branch and found this test that for some reason I thought is important:

[case testDataclassFineNewAttr]
from b import B
B(1, 2)
[file b.py]
from a import A
from dataclasses import dataclass
@dataclass
class B(A):
    b: int
[file a.py]
from dataclasses import dataclass
@dataclass
class A:
    a: int
[file a.py.2]
from dataclasses import dataclass
@dataclass
class A:
    a: int
    other: int
[builtins fixtures/list.pyi]
[out]
==
main:2: error: Too few arguments for "B"

[msullivan]

Aha, here is a test that needs some actual dependency handling:

[case testDataclassUpdate5]
# flags: --python-version 3.7
from b import B
B(1, 2)
[file b.py]
from a import A
from dataclasses import dataclass
@dataclass
class B(A):
    b: int
[file a.py]
from dataclasses import dataclass
@dataclass(init=False)
class A:
    a: int

[file a.py.2]
from dataclasses import dataclass
@dataclass(init=False)
class A:
    a: int
    other: int
[builtins fixtures/list.pyi]
[out]
==
main:3: error: Too few arguments for "B"

The others things I've tested were getting away without proper handling because __init__ would change and trigger __init__ in the subclass. But that needn't be true.