Visualising sceptre launches

[jeshan]

Using !stack_output is nice as it helps in reducing hardcoded outputs from other stacks. But having these dependencies means that some stacks will have to wait for others to update every time, even if they don't have any changes.
Currently, (I think) that there's no built-in facility to debug the call graph #553 .

I wanted a way to make it clear which are waiting on which so I built this:
https://github.com/jeshan/sceptre-dot

It outputs a graphviz dot file which can be rendered as png files.
There could be a command dot in the project but according to #201 and #524, there are no plans.
I would like the team to reconsider this as my solution requires only SceptrePlan and I don't think it will become a maintenance burden.

This will help:

1. address Add sceptre debug command #553
2. investigate slow deployments (as mentioned above)
3. provide visuals suitable to be added as documentation artefacts.

Let me know what you think about this.

[ngfgrant]

Hey @jeshan

Nice one. Yeh the comment in #201 was left before we implemented v2 fully. The implementation should be reasonably trivial in 2.0 thanks to using networkx under the hood.

We also had a text print out during development like:

# sceptre/config/graph.py

...

def __init__(self, stacks):
    ... 
    self.logger.debug("graph generated:\n\n{}\n\n".format(self)) # prints graph in debug mode

...

def __str__(self):
     return str(nx.nx_pydot.to_pydot(self.graph)) # will give a string representation


def __repr__(self):
    return nx.nx_pydot.to_pydot(self.graph) # will give a Dot object which could be read in to recreate graph

This was quite nice as it gave as text representation like:

strict digraph  {
x;
"eu-west-1/a/x";
"bundle/a/x";
"bundle/c/z";
"bundle/b/y";
"prod/eu/stack";
"bundle/c/z" -> "bundle/b/y";
"prod/eu/stack" -> "bundle/c/z";
}

Which you can feed into a visualiser such as http://www.webgraphviz.com/ or into pydot in the CLI.

What do you think about this approach?

[jeshan]

From what I can understand, your solution gives exactly the same result, correct? If so, I'd suggest we go with your approach.
This way, we'll be leveraging networkx to generate the dot output, unlike mine which just constructs dot strings (and probably contains yet-to-be-discovered bugs).

I'd suggest add a command like this to output the visualisation:

Usage: sceptre dot [OPTIONS] PATH

  Generates a DOT (Graphviz) representation of the Stack or StackGroup for a given config PATH.

Options:
  --help     Show this message and exit

[ngfgrant]

I think this could potentially fit well as a sub-command under sceptre generate... e.g. sceptre generate graph [stack]... thoughts? We could also default to printing the dot when in debug mode.

[jeshan]

agreed