Skip to content

Commit

Permalink
[engine] Introduce static analysis model and replace validator with it
Browse files Browse the repository at this point in the history
Introduces a Rule Graph that only contains the rules that cannot result in a Noop due to missing selector values. Replaces the existing validator with one backed by this new model. This means that for instance, intrinsics, or product graphs that depend on intrinsics, but are ultimately `Noop`ed can be eliminated early.

There's still a number of TODOs left in the patch, but it's to a point where I feel like I need feedback before making further changes.

Improvements
 * better error reporting over the previous implementation: The validator will now report transitive failures. Subject type match failures are also reported

Before
```
Found 1 rules with errors:
  (A, (Select(B),), noop)
    There is no producer of Select(B)
```
After
```
Rules with errors: 1
  (A, (Select(B),), noop):
    no matches for Select(B) with subject types: SubA
```

Doesn't cover
 *  cycles: I've got some ideas, but this has gotten pretty big already. Cycles won't cause non-termination because the graph generation algorithm only visits each rule/subject-type entry once.
 *  noops caused by tasks that can return `None`
 *  actually generating nodes base on the graphs
 *  variants
 *  HasProducts

Follow on work:
 * performance. There are improvements to the model generation that could be done. But, I want to split that work out from this initial cut.
 * integration into the scheduler. After this lands, I want to adapt the scheduler to use it for pre-filling dependencies of nodes to execute, and ultimately to simplify or eliminate SelectNodes.

Testing Done:
Introducing and running tests for the graph and validator locally. CI passed at https://travis-ci.org/pantsbuild/pants/builds/165056642

Bugs closed: 3890

Reviewed at https://rbcommons.com/s/twitter/r/4251/

closes #3890
  • Loading branch information
baroquebobcat committed Oct 4, 2016
1 parent ed80802 commit 9bb5959
Show file tree
Hide file tree
Showing 4 changed files with 1,106 additions and 148 deletions.

0 comments on commit 9bb5959

Please sign in to comment.