desi_job_graph makes job dependency and status graph webpage #1896
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a script that makes a webpage with the job dependency graph and status for a given night, e.g.
Clicking on a box takes you to the log for that job. This graph is based upon the processing table and SLURM job return codes; it does not check for missing job outputs like the dashboard and if any steps were run outside of the pipeline scripts that update the processing table, this won't know about them. Overall the dashboard is better for getting the big picture across many nights and for checking actual file outputs, while I find this view to be easier to identify what steps went wrong when there is a problem in a night, and for distinguishing "crashed" from "in progress".
Currently this is a single script for simplicity. If we find that we want to call it from other code (e.g. a spin webapp), it could be factored out into a separate module of desispec.