Skip to content

v2.8.0 — Ground truth

Choose a tag to compare

@Technologicat Technologicat released this 21 Aug 09:45
· 29 commits to master since this release

New

  • A parameter's type annotation is now treated as its type, so def f(obj: Thing): obj.method() draws an edge to Thing.method. A local already behaved this way — thing = Thing(); thing.method() always resolved — and an annotation is the one place a codebase states the type on purpose, which left the asymmetry hard to defend.
    • It is a static reading, and the value that arrives may be an overriding subclass. --ignore-parameter-annotations restores the previous behaviour, as does use_parameter_annotations=False in create_callgraph() and CallGraphVisitor.
    • Classes and modules bind — for both, the scope pyan resolves attributes against is the attribute namespace. A string annotation, Optional[X], or a union resolves to nothing.
    • *args / **kwargs do not bind, being a tuple and a dict, but subscripting either does — items[0] for *items: Thing, opts["k"] for **opts: Thing — since one element is what the annotation describes.

Fixed

  • Reading a module-level constant now reaches the constant, not its type. LIMIT = 42 bound the name to a Node standing for int, so a read drew an edge to int — never drawn, builtins not being in the analyzed set — and the constant, referenced by nothing, was dropped as unused. Most visibly this swallowed enum members: Color.RED is a name bound to a literal, so every member access resolved to str. Container constants were already correct, a dict or list literal having never bound a type.

  • A lambda inside a lambda no longer aborts the analysis. The inner scope was registered under a name the visitor never asks for, so any file containing one failed with ValueError: Unknown scope, taking the whole run with it — a single occurrence in a 328-file project meant no graph at all. The shape is ordinary: a test stub whose lambda builds an object from another lambda.

    • The same crash hit a lambda inside a comprehension on Python 3.12+, where PEP 709 stops symtable reporting the comprehension's table and moves the lambda's up to the enclosing function.
  • Calls made from a nested lambda or comprehension now reach the enclosing function. Folding an anonymous scope into its parent looked the parent up by splitting the namespace at its last dot — but an anonymous scope is named in two dotted pieces (lambda.0), so for anything nested the call was folded onto a node nothing else referenced and vanished from the graph. A function whose only call to helper went through two levels of lambda appeared to call nothing.

  • A call to an overridden method is no longer dropped when the caller also calls the base version. A postprocessing stage removed the more specific of two same-named uses edges whenever one class inherited the other, deleting a real call site: a function calling both ASTMarker(...) and a locally defined Tagged(ASTMarker) lost its edge to Tagged.__init__. Attribute lookup returns one target per call site, and the stage excluded wildcards by construction, so it could never reach the ambiguous case it was written for — it has been removed.

Changed

  • Analyzing a large project is much faster — 49s to 16s on a 94k-line, 328-file codebase, with byte-identical output. In the analyzer, what was a linear scan over the whole graph is now an O(1) table lookup.

  • Uses edges that a more specific edge already conveys are now dropped. A bare import produces a uses edge whether or not the name is ever referenced, so a module node accumulated one per imported name — and in a graph that also shows functions, each ran parallel to the edges of the functions using it, from a node drawn right beside them. Two shapes are culled: a module's edge to a target that a function or class defined in that same module also uses, and a module's edge to another module it also reaches into. Only uses edges — a module's defines edges to its own functions are never affected. (#140)

    • Edges that nothing else records are kept — a module-level use no function reproduces (router = make_router()), and an import whose name is never referenced.
    • A module-level view is unaffected: under --depth 0 the finer edge collapses back onto the same pair, so no dependency disappears.
    • --keep-subsumed-edges restores the raw edge set, as does cull_subsumed_edges=False in create_callgraph() and CallGraphVisitor.
  • A grouped graph no longer draws each module twice. Under --grouped / --nested-groups, a module with members was drawn both as the cluster holding them and as an ellipse beside it carrying the same label. The cluster is the module, so the node now sits inside it, labelled <module> after the module-level code object CPython names the same way. (#140)

    • Every module is drawn as a box, whether or not it has members, so a module's appearance no longer depends on its contents. A package with an empty __init__.py gets a box holding <module> alone.
    • <module> itself appears where the module body uses something, where something uses the module, or where it is the only node its box would hold. A module that merely contains definitions is left to its members.
    • A module's defines edges to its own members are no longer drawn either, the box having already said it.
    • Ungrouped output is unchanged: module nodes keep their dotted names and their defines edges, which are then the only indication of what contains what.
  • The rules governing what pyan omits from a graph are now documented, under What the graph leaves out in the README.