Create a graph of all plugins. #1777

bangerth · 2017-05-25T16:27:37Z

Here is an attempt at addressing #1667. It adds a command line option that, when
given, initializes a Simulator object and then systematically goes through all
plugin systems and asks them to write out their part of the connection graph
for all plugins.

I haven't gone through all systems yet, just seven of them, to see whether
this is the right direction to go. It's a bit of work to convert it all,
so I thought I'd put it out there already and get a bit of feedback.

bangerth · 2017-05-25T16:29:18Z

This is the graph I have so far. It's going to get big once we have all of the subsystems, but I think it's still an interesting exercise. It also shows that basically everything by now derives from SimulatorAccess, which I think is a bit of a shame...

bangerth · 2017-05-25T16:30:57Z

I should also add why I put writing the output into the Simulator class, rather than calling all of these static functions from main.cc: I have vague plans to eventually color those plugins that are selected for a particular run in a different color. But that's a later step.

bangerth · 2017-05-25T17:10:32Z

Now slightly nicer with roughly equally sized blue circles:

ljhwang · 2017-05-25T19:57:36Z

Nice visual. Great way to see everything! Best, -Lorraine ***************************** Lorraine Hwang, Ph.D. Associate Director, CIG 530.752.3656

…

On May 25, 2017, at 10:10 AM, Wolfgang Bangerth ***@***.***> wrote: Now slightly nicer with roughly equally sized blue circles: <https://cloud.githubusercontent.com/assets/7504421/26461569/c14feb6c-413a-11e7-9f06-ec69858e43a1.png> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1777 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AESQX5k5QCWqpafatxbltTE9ANZTnJOMks5r9bYIgaJpZM4NmmB0>.

gassmoeller

Found the mandatory typo 😄

I like it! Makes it much easier to understand the structure of ASPECT. It is a bit of a shame that we need to introduce another boiler-plate function into every interface, but with the plugin list in an anonymous namespace I also do not see a way around it.

gassmoeller · 2017-05-26T14:57:43Z

source/simulator/helper_functions.cc

+        "        style=\"filled\"];\n"
+        "\n";
+
+    // then also write nodes for the Simulator and SimulatorAccess clsses,


bangerth · 2017-05-26T15:12:53Z

OK, if you think this is useful, then I will continue with this quest.

bangerth · 2017-05-27T03:58:18Z

The latest picture -- it's starting to get a bit busy...

bangerth · 2017-05-29T04:08:57Z

Here is the final version of it all. It is pretty mechanical, there are really only 3 parts that are interesting:

The changes to plugin.h that provide the actual functionality we use at the bottom, along with the top-level magic in helper_functions.cc (which are called from a code path in main.cc)
The fact that unlike all other plugin systems, visualization postprocessors do not feed into Simulator, but into Postprocess::Visualization instead.
The fact that I needed to change the HDF5 particle output class in such a way that we can now generate such objects even if HDF5 is not actually supported. This is because we register this plugin whether or not we support HDF5. (I could also just have disabled registration of that plugin, but in that case you'd get an error message about the fact that there is no hdf5 plugin, whereas with the current system you get an error message that explains why there is such a plugin, but you can't use it.)

Other than that, it's just a bunch of highly repetitive stuff. I hope to remove some of this again when merging the various Manager classes, see #1775.

bangerth · 2017-05-29T04:10:44Z

Final version of the graph, with all plugins we have (all 223!):

gassmoeller

It looks like this is getting really big. I am not sure a simple figure in the manual will be readable. Did you think about where to put the product? Otherwise: Good job! Nearly ready.

gassmoeller · 2017-05-29T20:34:28Z

source/particle/generator/interface.cc

+      void
+      write_plugin_graph (std::ostream &out)
+      {
+        std_cxx11::get<dim>(registered_plugins).write_plugin_graph ("Particle property interface",


I think this should be "Particle generator interface"

Good catch!

gassmoeller · 2017-05-29T20:43:52Z

source/simulator/helper_functions.cc

+#include <aspect/particle/interpolator/interface.h>
+#include <aspect/particle/output/interface.h>
+#include <aspect/prescribed_stokes_solution/interface.h>
+#include <aspect/postprocess/visualization.h>


In simulator_access.cc we assume that all of these are included in simulator.h and therefore do not list them explicitly. Is there a reason you want to handle this differently here?

That's not quite true. For example, all of the particle headers are not included in simulator.h, nor are the viz header file. You are correct about the prescribed_stokes_solution header, though, as well as possibly a few of the others.

In general, I think it's useful to really include in a file what is needed, rather than relying on some other header doing so transitively. What do you want me to do?

I'll add that there is essentially no cost associated with including a file twice -- the compiler caches which files it (i) has included already, and (ii) that are guarded by include guards.

I was just curious. Let us keep it as is.

bangerth · 2017-05-29T21:51:39Z

I've updated the patch.

What to do with the graph: I definitely want to use it in presentations, where I can also zoom in to smaller regions. I've also thought about putting it into the manual -- not so that one can actually read (without zooming in) what the graph says, but to illustrate the architecture of ASPECT.

With the infrastructure in place, it would not be very difficult to make things work in such a way that you could pass an argument to the command line that lists which subsystem you want to see. This would lead to smaller, more targeted graphs. I think some experimentation might be necessary to get this right.

bangerth · 2017-05-29T21:53:46Z

An idea that's way out there: One could imagine a tool where you get to see the graph and you can click on individual circles to enable them in your simulation. If you right clicked, you could set the parameters that correspond to this option. I'm not volunteering to write such a tool, but it would be cool way to do graphically what your GUI already does :-)

gassmoeller · 2017-05-30T15:52:14Z

Looks good.

tjhei

I am a little late to the party, but I have comments/questions:

Neat!
Would it make sense to use the name used in the ParameterHandler section in addition to the name of the classes? I know that we don't have an automatic way to identify this name right now, but we talked about it before: automatic plugin prm subsection name #97
I created a test: add write_plugin_graph test #1795

tjhei · 2017-05-31T14:05:12Z

source/simulator/helper_functions.cc

+    out << "digraph Plugins\n"
+        "{\n"
+        "  splines=line;\n"
+        "  splines=true;\n"


why do you need two assignments to splines? Isn't line=false?

Good question, I don't know either, but it makes a difference and that's what the online documentation also seems to suggest one uses. Don't ask me...

tjhei · 2017-05-31T14:08:24Z

source/simulator/helper_functions.cc

+        "{\n"
+        "  splines=line;\n"
+        "  splines=true;\n"
+        "  overlap=false;\n"


Can you add layout=neato;? Without it, the graph is way too wide.

I convert the file using neato. Adding the line doesn't seem to make any difference to me -- what are you using to create the .png?

dot -Tpng bla.dot >bla.png

Ah. Just use the neato command line tool. I could also add the layout statement if you want.

Now in #1802.

bangerth · 2017-05-31T15:09:45Z

On your second point: right now I use the name of the plugin that is passed to the set Xxx model = ... parameter. This is at least one canonical name the users will see. Generally, of course, the parameter section name is the same with just the first word capitalized.

I think the name of the plugin is the correct one to output as the primary piece of information because not all plugins actually read parameters from any section (think for example of the various -- in fact many :-) -- visualization plugins).

bangerth force-pushed the plugin-graph branch from c92ad88 to 7212d05 Compare May 25, 2017 17:09

bangerth mentioned this pull request May 25, 2017

Remove SimulatorAccess base class from plugins that don't need it #1778

Closed

gassmoeller reviewed May 26, 2017

View reviewed changes

bangerth force-pushed the plugin-graph branch from 7212d05 to 80f9508 Compare May 29, 2017 04:04

bangerth changed the title ~~[WIP]Create a graph of all plugins.~~ Create a graph of all plugins. May 29, 2017

bangerth force-pushed the plugin-graph branch from 80f9508 to 0fb4cea Compare May 29, 2017 04:12

gassmoeller reviewed May 29, 2017

View reviewed changes

Create a graph of all plugins.

450c3f5

bangerth force-pushed the plugin-graph branch from 0fb4cea to 450c3f5 Compare May 29, 2017 21:48

gassmoeller merged commit ed89a9d into geodynamics:master May 30, 2017

bangerth deleted the plugin-graph branch May 30, 2017 15:59

bangerth mentioned this pull request May 30, 2017

Create, update and use the graph of all plugins #1787

Closed

tjhei mentioned this pull request May 31, 2017

add write_plugin_graph test #1795

Merged

tjhei reviewed May 31, 2017

View reviewed changes

This was referenced Jun 1, 2017

Explicitly use neato layout for the plugin graph. #1802

Merged

Update the style for the plugin graph. #1804

Merged

Sort plugins before outputting them as a graph. #1806

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create a graph of all plugins. #1777

Create a graph of all plugins. #1777

bangerth commented May 25, 2017

bangerth commented May 25, 2017

bangerth commented May 25, 2017

bangerth commented May 25, 2017

ljhwang commented May 25, 2017 via email

gassmoeller left a comment

gassmoeller May 26, 2017

bangerth commented May 26, 2017

bangerth commented May 27, 2017

bangerth commented May 29, 2017

bangerth commented May 29, 2017

gassmoeller left a comment

gassmoeller May 29, 2017

bangerth May 29, 2017

gassmoeller May 29, 2017

bangerth May 29, 2017

bangerth May 29, 2017

gassmoeller May 30, 2017

bangerth commented May 29, 2017

bangerth commented May 29, 2017

gassmoeller commented May 30, 2017

tjhei left a comment

tjhei May 31, 2017

bangerth May 31, 2017

tjhei May 31, 2017

bangerth May 31, 2017

tjhei Jun 1, 2017

bangerth Jun 1, 2017

bangerth Jun 1, 2017

bangerth commented May 31, 2017

Create a graph of all plugins. #1777

Create a graph of all plugins. #1777

Conversation

bangerth commented May 25, 2017

bangerth commented May 25, 2017

bangerth commented May 25, 2017

bangerth commented May 25, 2017

ljhwang commented May 25, 2017 via email

gassmoeller left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bangerth commented May 26, 2017

bangerth commented May 27, 2017

bangerth commented May 29, 2017

bangerth commented May 29, 2017

gassmoeller left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bangerth commented May 29, 2017

bangerth commented May 29, 2017

gassmoeller commented May 30, 2017

tjhei left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bangerth commented May 31, 2017