Static graphviz task visualisation

[Casyfill]

I am using a small code I wrote for static visualization of Task DAGS using graphviz lib. It is very basic, but still quite useful for documentation purposes - I even made a habit of rerendering DAGS into task README.md on commits.

I will be happy to share my code via PR, but was wondering if core contributors

1. think it is a good idea overall
2. have any opinion on where to put it, e.g. through luigi.tools.staticviz or luigi.contrib.staticviz...

[dlstadther]

I'm of the option either

• luigi.tools if we want to keep this tightly coupled, or
• as a separate, independent github project

Could you provide some insight into how it is executed against a luigi pipeline?

[Casyfill]

Hey @dlstadther, sounds good. I made a [premature] PR at #2328 with staticviz as part of luigi.tools.

And the approach is quite simple - for now, I provide my function with an endpoint task (last task in the graph), and it scrolls up using required method of each consequent node (task). Of course, this approach is somewhat limited, as often we have multiple possible outcomes (thus, multiple endpoint tasks) for one pipeline. I hope to mitigate that in the future, probably just by creating a separate graph for each endpoint, and then merging the graphs (we can use different edge colors/styles for different pipelines, though).

but before, I have to figure out what went wrong with the tox and cover the whole thing with tests

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If closed, you may revisit when your time allows and reopen! Thank you for your contributions.

[EngineerReversed]

Yes, a static graph visualization would really help. We are working on HPC box and don't have luxury to host servers to visualize graph. A static dump of the dependency graph using graphviz would surely help.

[Casyfill]

Let me pull and brush my code then. It wasn’t perfect, but worked for me

…

On Thu, Dec 31, 2020 at 09:40 Abhishek Shakya ***@***.***> wrote: Yes, a static graph visualization would really help. We are working on HPC box and don't have luxury to host servers to visualize graph. A static dump of the dependency graph using graphviz would surely help. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2328 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAUNHJJ5BB3Q6CFUXJPBAJDSXQMEHANCNFSM4ELML24Q> .