Scope graphs

[robrix]

This PR defines a draft of a scope graph analysis, accumulating values representing the declarations and references in a program. This is imperfect in a number of ways which I will enumerate in a review, below, but I’d like to get it merged in as an initial approach which we can contrast with some improved plans.

• Depends on Fold over Core without iter #200.
• Depends on Types as syntax #201.
• Depends on 🔥 Naming, Name, & Gensym #202.
• Depends on Records #203.
• Depends on Sequence values in the abstract domain #204.
• Depends on Factor annotations out of Core #208.
• Depends on Generalize analyses over the term type #209.

Scope graph output for one of the example programs

ScopeGraph
  { unScopeGraph =
      fromList
        [ ( Decl
              { declSymbol = "_x"
              , declLoc =
                  Loc
                    { locPath = "src/Analysis/Eval.hs"
                    , locSpan =
                        Span
                          { spanStart = Pos { posLine = 117 , posCol = 20 }
                          , spanEnd = Pos { posLine = 125 , posCol = 5 }
                          }
                    }
              }
          , fromList
              [ Ref
                  Loc
                    { locPath = "src/Analysis/Eval.hs"
                    , locSpan =
                        Span
                          { spanStart = Pos { posLine = 119 , posCol = 13 }
                          , spanEnd = Pos { posLine = 119 , posCol = 28 }
                          }
                    }
              ]
          )
        , ( Decl
              { declSymbol = "_y"
              , declLoc =
                  Loc
                    { locPath = "src/Analysis/Eval.hs"
                    , locSpan =
                        Span
                          { spanStart = Pos { posLine = 117 , posCol = 20 }
                          , spanEnd = Pos { posLine = 125 , posCol = 5 }
                          }
                    }
              }
          , fromList
              [ Ref
                  Loc
                    { locPath = "src/Analysis/Eval.hs"
                    , locSpan =
                        Span
                          { spanStart = Pos { posLine = 120 , posCol = 13 }
                          , spanEnd = Pos { posLine = 120 , posCol = 28 }
                          }
                    }
              ]
          )
        , ( Decl
              { declSymbol = "mkPoint"
              , declLoc =
                  Loc
                    { locPath = "src/Analysis/Eval.hs"
                    , locSpan =
                        Span
                          { spanStart = Pos { posLine = 117 , posCol = 20 }
                          , spanEnd = Pos { posLine = 125 , posCol = 5 }
                          }
                    }
              }
          , fromList
              [ Ref
                  Loc
                    { locPath = "src/Analysis/Eval.hs"
                    , locSpan =
                        Span
                          { spanStart = Pos { posLine = 122 , posCol = 41 }
                          , spanEnd = Pos { posLine = 122 , posCol = 61 }
                          }
                    }
              ]
          )
        , ( Decl
              { declSymbol = "x"
              , declLoc =
                  Loc
                    { locPath = "src/Analysis/Eval.hs"
                    , locSpan =
                        Span
                          { spanStart = Pos { posLine = 118 , posCol = 66 }
                          , spanEnd = Pos { posLine = 121 , posCol = 7 }
                          }
                    }
              }
          , fromList []
          )
        , ( Decl
              { declSymbol = "y"
              , declLoc =
                  Loc
                    { locPath = "src/Analysis/Eval.hs"
                    , locSpan =
                        Span
                          { spanStart = Pos { posLine = 118 , posCol = 66 }
                          , spanEnd = Pos { posLine = 121 , posCol = 7 }
                          }
                    }
              }
          , fromList []
          )
        ]
  }

[robrix]

Ready for review.

Note that we’re likely to want to compose scope graph computation with the computation of e.g. types, so that we can refine scopes based on type information: e.g. if f only gets called with an X, then methods called on its parameter can be looked up on X.

That plus the shortcomings noted below lead me to believe that a better solution in the long run will be for us to treat scope graph computation as an out-of-band accumulation of a graph which composed freely onto other analyses. My current plan for this is based on the abstract garbage collection described in Abstracting Definitional Interpreters, where a simple root-calculating interpreter with knowledge about the syntax it’s evaluating is composed onto the basic interpreter.

This should yield simpler computations of exhaustive scope graphs, and also allow us to tune the precision of the calculation via our selection of another abstract domain.

[robrix]

We were leaving record fields uninitialized in the abstract analyses, which meant that later lookups would fail.

[robrix]

This model of scope graphs has no recursive structure; references map directly to declarations, and not e.g. through other declarations in a path.

This makes this scope graph strictly less informative than those we’re computing in semantic proper, tho it’s hard to say for sure if that makes it strictly less useful for our purposes.

[robrix]

Unlike ImportGraph, our ScopeGraph abstract domain doesn’t require a semi-concrete representation of values alongside it, because we don’t need to treat any of the primitives specially: ImportGraph needs strings and closures modelled explicitly so that it can notice loads issued within methods like Ruby’s require, but ScopeGraph doesn’t require any special treatment of primitives (other than to construct a scope graph).

[robrix]

Binding remembers (locally) the location for the binding site. Unfortunately, this will generally be the location (and especially the Span) of the Ann node surrounding the Lam or :>>=, making it a little imprecise. This could be improved by adding (optional) locations to these nodes for the bound variables, and since Core is curried, we’d do the right thing for multiple parameters automatically.

[patrickt]

👍 File an issue about this?

[robrix]

I’m actually just tackling it in a follow-up PR.

[robrix]

Because we’re modelling scope graphs as an abstract domain, we’re limited to producing them in the few methods which take or return a value, notably deref, assign, and record. In particular, we might prefer to build them in lookupEnv or ... in order to preserve nested structure in the latter, but that would require out-of-band scope graph construction e.g. via a Writer effect.

[patrickt]

Can we pull out (foldMapA (pure . Just . mappend (extendBinding addr ref bindLoc))) into a let-bound function with a name? It’s a bit intimidating as is.

[robrix]

Fixed in 0515d7c.

[robrix]

bindLoc might be Nothing if e.g. we’re attempting to read a free variable, or if we’re trying to look a name up outside of its lexical binding scope—indirectly, using records. Thus, record field accesses don’t currently get added to the scope graph, which limits the utility of this approach.

[patrickt]

One small request for clarity, but other than that this is fantastic work!

[patrickt]

This is so readable 😍

[patrickt]

Can we pull out (foldMapA (pure . Just . mappend (extendBinding addr ref bindLoc))) into a let-bound function with a name? It’s a bit intimidating as is.

[patrickt]

I’m normally an anti-RecordWildCards partisan, but this is a 100% copacetic use thereof.

[robrix]

Yeah, it’ll do until such time as we replace these with effects.

[patrickt]

👍 File an issue about this?

[patrickt]

I think this varies based on language? Not sure, though.

[robrix]

Shouldn’t for the types, but might for values.