Visualize a Phlex graph as GraphViz "dot" (or others).

[brettviren]

With Wire-Cell Toolkit's use of the data-flow processing pattern, it has been almost indispensable to generate GraphViz dot representations of the DFP graph as we develop a configuration. The rendering makes immediate any bugs in how the graph is wired and also serves as powerful documentation for the "shape" of the job. The rendered graph is also a tool to reason about issues for a given workflow.

It would be great if we could do similar with Phlex DFP graphs.

With WCT, the DFP graph is fully specified by the configuration and so it is relatively simple to render. The wcpy dotify command from wire-cell-python provides one render method.

With Phlex, things are rather less simple. Here are things that defeat my attempt to make a dotify type command for Phlex graphs, compared to WCT graphs:

1. The algorithm nodes are not wired directly but through data products.
2. The data product produced or consumed are specified by an incomplete query which is interpreted according to heuristics defined in the Phlex C++.
3. Some data products and entire data-layer hierarchy layers are known only to Phlex runtime.

The first two can be overcome by replicating some Phlex C++ logic in the render command. The second needs (I think) actual algorithm instantiation.

Do I have the situation properly understood?

Are there other problems to solve to make such a graph renderer?

Would it be best done as a phlex subcommand or a special C++ main() using libphlex?

Here's my current attempt which does seem to work okay getting past problems 1 and 2. I don't yet have a fold/unfold or other more dynamic Phlex graph building examples.

https://github.com/brettviren/graphlex

[brettviren]

I added this feature as a new program graphlexed in my Phlex bag of hacks. Since it is a C++ main() it can actually construct the actual Phlex graph before translating it to GraphViz.

https://github.com/brettviren/phlexed

To make it work I had to use a hack to make public the nodes_ and fixed_hierarchy_ members of framework_graph, itself in the phlex::detail namespace. If user code is deemed worthy of accessing this information, the change is minimal (ignoring the namespace issue):

diff --git a/phlex/core/framework_graph.hpp b/phlex/core/framework_graph.hpp
--- a/phlex/core/framework_graph.hpp
+++ b/phlex/core/framework_graph.hpp
@@ -63,6 +63,15 @@ namespace phlex::detail {

     void execute();

+    // Read-only access to the populated node catalog and fixed data-layer
+    // hierarchy after load_module()/load_source()/load_driver() but without
+    // executing the graph.  Intended for introspection / dry-run tooling
+    // (e.g. rendering the data-flow graph).  The returned references are valid
+    // for the lifetime of this framework_graph.
+    [[nodiscard]] node_catalog const& nodes() const noexcept { return nodes_; }
+    [[nodiscard]] fixed_hierarchy const& hierarchy() const noexcept { return fixed_hierarchy_; }
+
     std::size_t seen_cell_count(std::string const& layer_name, bool missing_ok = false) const;
     std::size_t execution_count(std::string const& node_name) const;