Workflow Comments 💬

[ElectronicBlueberry]

What are Workflow Comments 🤔

Workflow Comments are a new feature coming to the Galaxy Workflow Editor. They add a suite of tools to help you visually to explain and visually structure your Workflows.

Comments are saved on your Workflow, so they can be shared with other Workflow contributors, help guide workflow users, or just help you keep track of your work and sort your thoughts, while developing a workflow. They can also help you with teaching, live demos, and providing feedback on a workflow, all directly inside the Workflow editor.

New Tools 🛠️

One of the first noticeable changes is the addition of a new UI element to the workflow editor: the Editor Toolbar.

This new feature allows you to select tools other than the mouse pointer to edit and decorate your Workflow. In this update, the new features present in the Toolbar mostly center around the new Workflow Comments (with exception of the cursor and magnet), but additional general editor tools may be implemented in the future.

Snapping 🧲

Our first new editor tool can be used on Steps as well as Comments. The snapping editor tool is a toggle, and when turned on, you will notice a new option appear right next to the Toolbar.

Whenever this option is shown while using an editor tool (such as the cursor), snapping is active. The slider can be used to adjust the strength of the snap. Using snapping makes it super easy to align your Steps and Comments to the Workflow gird!

Comments 🖋

The next set of tools all have to do with Workflow Comments. Let's look at the types of Comments you can place, and the tools you need to do so.

Text Comments 🔤

The first type of Comment is a simple Text Comment.
Text Comments are just free floating text, which can be styled. It's main uses include section headings and short comments.

To place a Text comment, select the Text Comment Tool, then drag a box which determines where your new Comment will be placed. Finally begin typing, to change the Text in the comment.

2023-10-24.17-27-15.mp4

In the menu that pops up when a Comment is selected, it can be Styled in multiple ways, making the whole text italic, bold, coloured, or larger / smaller.

You can also change the default settings before creating a Text Comment in the top bar, while having the Text Comment editor tool selected.

Markdown Comments 📝

Markdown Comments also display text, but as rendered markdown. Editing the text will display the raw text, and selecting anything else will show the Comment in a rendered state. They also come with a border and background, and scroll if they overflow.

The main use for Markdown Comments is leaving longer, rich-text, explanations or notes.

Most of the styling for this type of Comment is done with markdown itself, but just like all other Comments they can be assigned a colour, colouring the Comments border and contents.

Place a new Markdown Comment by selecting the Markdown Comment tool, and dragging a box.

Frame Comments 🔲

Frame Comments are a bit different from the two types of Comments we just looked at. They can only display a short bit of text at the top, and otherwise consist of a faintly coloured box. They are used to visually group parts of your workflow together, Steps and Comments alike.

Have a look at this example, showing a beautifully structured structural workflow:

Quick Tip: If you're wondering how to make a screenshot of your workflow with a resolution this high:

• Using Firefox, open your Workflow
• Open the console by pressing ctrl+shift+i, and then selecting the console tab
• Dock it to the side, so it dosen't reduce your screen height too much, by pressing the three dots on the right
• Type :screenshot --dpr 6 and hit enter. The number behind the --dpr option determines by how much the image is upscaled from your browsers resolution

Placing Frame Comments is easy. Just select the frame comment tool, and drag around the Steps / Comments you want grouped in a frame. Don't worry about the exact placement too much, as you can use the "fit to content" button, to automatically resize the frame to fit your content with a comfortable margin.

2023-10-19.16-22-02.mp4

Frame Comments also offer a convenience feature, which can help you in quickly re-arranging your Workflows. When a Frame Comment is moved, all it's contents stick to it, and are moved along with it.

2023-10-26.14-37-31.mp4

The grouping hierarchy of frame comments is saved on the workflow, so using Text- / Markdown Comments together with Frame Comments, can make it easier for other software to understand your Comments, if they implement support for Workflow Comments.

A word of caution: When importing your Workflows to Software other than the newest Galaxy version, Comments can be lost. Your Workflows will still function perfectly fine, since Workflow Comments are optional and the rest of your Workflow remains backwards compatible when using them.

Freehand Comments 🖍

Finally, there are Freehand Comments. These come with not one, but two new Editor Tools: the Freehand Pen, and the Freehand Eraser.

Freehand Comments are drawings directly on your Workflow. The Freehand Pen has configurable smoothing, so these Comments are fairly easy to draw with a mouse. They are perfect for showing what you are referring to in live demos, leaving quick marks when reviewing or debugging a Workflow, drawing arrows to show what your Markdown Comment is referring to, etc. Also doodles.

Removing Freehand Comments is just as easy as drawing them. Either use the Freehand Eraser, or press the "Remove all" button in the Editor Tool Bar which appears when either the Pen or the Eraser is selected

Minimap 🗺️

It is also worth mentioning that all these Comments, and their colours, are rendered to the Minimap, in the lower right hand corner of the Workflow Editor, as can be seen in several examples above. This can make the Minimap much more useful in quicky getting your bearings on where you are in the Workflow, especially when making use of Frame Comments.

---

We hope you find Workflow Comments a useful addition to your Workflow workflow, no matter how you may work with them. We're also excited for the possibilities the new Editor Tool Bar offers for the Editor in the future.

As always, please let us know if you have any feedback about this feature.

Thanks for reading!

License

• I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

[martenson]

This looks absolutely marvelous.

[jmchilton]

These are brilliant and beautiful and terrifying.

I worry about the maintenance burden of putting so much display logic into the workflow definition. I'll admit my biases here though - human writable workflows without the editor has been my poorly executed goal for a decade - you've done so much more to make workflows accessible than I have in so much less time. None of these comments is even close to a -1 on this and none of this means I'm not wildly impressed by this implementation.

Does the grouping annotation decide what is grouped by the (x,y) position of the node in the workflow definition or by the rendered box? The grouping annotation seems to convey some semantic value so I'd love for it to be keyed on data encoded in the workflow definition and not having anything to do with the rendered box sizes - those have changed over time.

I've also seen the workflow editor change styles dramatically over the years in a way that makes me very anxious about the colors. Group annotations and text/markdown annotations without color selection... would make me so much less anxious. I would drop the freehand annotations all together.

We have a separate display implementation of the workflows in terms of Cytoscape (https://github.com/galaxyproject/gxformat2/blob/master/gxformat2/cytoscape.py / https://js.cytoscape.org/). Features that we can potentially port to that display implementation make me feel a lot better. It tells me that our abstraction description can be displayed in multiple ways... and that is a good indication the maintenance burden isn't overwhelming and we're not .... overfitting the JS tools we have available to us. I'm not insisting you have to update that code or find the implementation.... but if we could bring these features in in small bits and give me some time to find the cytoscape implementation as we bring them in... again that would make me feel less anxious. But maybe none of this is viable in that implementation - I should check it out and see.

[jmchilton]

Also we already have something called workflow annotations - I would suggest we use a different name if that is okay. Maybe call them workflow comments or workflow markups?

[ElectronicBlueberry]

Does the grouping annotation decide what is grouped by the (x,y) position of the node in the workflow definition or by the rendered box? The grouping annotation seems to convey some semantic value so I'd love for it to be keyed on data encoded in the workflow definition and not having anything to do with the rendered box sizes - those have changed over time.

I purposefully kept this feature void of semantic value. The grouping annotation is meant only as a visual hint, that conveys that this part of the workflow belongs together in some sort of way. Essentially it is a purely visual box drawn around a part of the workflow. The moving of the nodes together is an ease of use feature, and it is determined at runtime.

I should have detailed the intended use of this feature in the PR description. I'll post it instead here:

My thought process for keeping these annotations (or comments/markups) purely visual is to not break the semantic representation of a workflow, in that they should never be able to influence how a workflow is handled outside of the editor itself. Instead I want these to serve as a visual aid to help teach, develop and communicate workflows.

Especially the freehand annotations are not meant to be a permanent part of a workflow, but a quick and easy communication tool, which is why they are bulk-deletable.

A big inspiration for this was the Blender node editor, which has a very comparable set of features it treats in the same way. There, these are most often seen in tutorials, example files, and during collaborative development.

I've also seen the workflow editor change styles dramatically over the years in a way that makes me very anxious about the colors. Group annotations and text/markdown annotations without color selection... would make me so much less anxious.

I think the colours are most important for groups, as they support their core feature of visually differentiating parts of a workflow at a glance. As for the workflow editors colours chaning in the future: the colours are not stored in the database, but defined client-side. The annoation only stores which colour was selected, not how that colour is rendered, so that future colour changes and theme support can adapt them to match the editors look. This is why they are only selectable from a preset of colours.

Also we already have something called workflow annotations - I would suggest we use a different name if that is okay. Maybe call them workflow comments or workflow markups?

Yes, I think calling them comments may be a better to avoid confusion with other types of annotations.

[jmchilton]

I understand your position on the colors - I think I just don't agree on priorities. I'd call grouping nodes together a semantic thing... but again we can disagree on language - that is not a big deal.

Detailing use cases would be awesome - but to me the use case for notes and grouping is so clear and I definitely want it generally. I am just anxious about an implementation that is so tied to the editor. Just throwing editor state into the database is such a problem for me - we're still dealing with random unstructured decisions from 15 years ago that started that way.

If you think the ease of implementation is important and want to leverage runtime state this way... another proposal that would completely eliminate my anxiety is ... like clicking a button in the editor called screenshot builder and giving the user a transient copy of the workflow that they can markup this way and take screenshots of for tutorials and just never save any of it back to the workflow definition/database. I suspect this would annoy users... but making it clear it is for screenshots might alleviate that. Is this just splitting the baby in such a way that no one is happy? I would certainly be sad not to have persisted notes and grouping.

[mvdbeek]

I can certainly add another option noone but me will like: If you draw a rectangle and you cover more than one node you push those boxes transparently into a subworkflow, and then you annotate the whole subworkflow. if you only cover one node then you annotate just that one node. This though is probably a lot of additional work, so ... consider this only if we're all on the same page about that idea.

[ElectronicBlueberry]

I can certainly add another option noone but me will like: If you draw a rectangle and you cover more than one node you push those boxes transparently into a subworkflow, and then you annotate the whole subworkflow. if you only cover one node then you annotate just that one node. This though is probably a lot of additional work, so ... consider this only if we're all on the same page about that idea.

If making groups semantic is the way we want to go, I actually quite like this. I find the UX on this quite complicated though, as it raises several difficult questions:

• how is this happening communicated to the user?
• how to take nodes back out of a group / sub workflow?
• what happens to connections which go out of, and then back into a group?

Similar questions also arise if groups become semantic without creating sub-workflows. The PRs current state of being purely visual is I think the simplest from a user facing side, even if it dosen't cover the use case of data-structrual grouping.

[jmchilton]

@mvdbeek's idea is very far a field from this work but also I think would be amazing. Making subworkflows easier to extract would be lovely, adding like a Markdown or Plain text note that describes the role of subworkflow as an attribute in the parent workflow would make perfect sense both in the workflow editor and in the YAML version of the workflow. Editors and visualizers could handle this however they wanted.

[jmchilton]

Extending the @mvdbeek suggestion... subworkflow nodes could have a display attribute - display just the node, display the node and the note about it, or display both and a RO version of the workflow itself - you could then get something that kind of vaguely looks like the first screenshot. How to render the note and the workflow in a compact space together automatically would be a hard problem... but not un-fun... or maybe it would be?

[mvdbeek]

• what happens to connections which go out of, and then back into a group?

and I assume we don't want pull those intermediate connections into the subworkflow ? that's tricky, I don't have an answer for that ... the other two items are also difficult but potentially doable.

[jmchilton]

how is this happening communicated to the user?

When you create a group using the "extract subworkflow" tool with a clever icon and useful tool tip - a modal pops up and describes the nodes being extracted and giving the user the option to label the inputs and outputs being created.

how to take nodes back out of a group / sub workflow?

You don't I guess?

what happens to connections which go out of, and then back into a group?

Don't allow extractions on these - it isn't a subworkflow.

I'd love if this was implementation via the workflow refactoring API as an atomic refactoring step. I'd implement the backend and give you a nice pydantic action model for describing what is being extracted.

[jmchilton]

(These are different ideas I think both worthy of discussing independent of each other.)

[ElectronicBlueberry]

In terms of UX, that all sounds much more complicated than the user interface implemented in this PR.

I get the feeling from this discussion that we are approaching this from two very different angles.

My opinion on this in a broader sense is that the workflow editor is a type of visual programming, and therefore inherently visual. For this reason we will never be able to get around storing some visual state if we want to make full use of it. I understand the usefulness of displaying the data structure behind a workflow in different ways, such as with cytoscape, but think that entangling this with the visual aspects of how the editor renders workflows limits the workflow editors potential.

My intention for this feature was not to extend or rework the semantics of workflows, but extend the editor in a way that leans into the visual aspect of the workflow editor, giving users new means to visually structure, explain, and display their workflows, while developing or viewing them within galaxys workflow editor.

[jmchilton]

A totally valid perspective I completely respect - I just take the other side of coin very firmly. Constraining the workflow editor into things have multiple implementations makes Galaxy less of a walled garden, makes Galaxy more declarative and accessible, helps our long term sustainability and maintainability, etc.. The most important features Galaxy workflows need is a solid text editor with state validation... not more visual state.

To me workflows are not a visual programming language - they are a sequential programming language with a nice visualization component.

So I totally agree we are 100% approaching this from different angles - but I think there is a lot of room to meet in the middle and do a lot great work that helps both approaches. I understand the compromises would be frustrating though...

[abueg]

this feature looks super promising!! i can see this being really helpful for the VGP workflows, since so much of the visually dense workflows (as seen in the editor) can be thought of as different parts running parallel (eg, preprocessing, actual assembling, various QC functions), i think it'd definitely be helpful if these comments can stick with the workflow and users can see all that info when they import the workflow and use it on their own data. it makes it a bit less of a black box (or i guess a navy and white box lol), which is easier to teach/visualize 🥰

[ElectronicBlueberry]

@jmchilton Regarding the issue with the colours: How would you feel about instead of these comments storing a colour, they would store a more general 'variant', which would just be an enumerated value. How different variants are displayed is then entirely up to the implementation. Different variants could be displayed as an arbitrary set of colours, outlines, symbols, or just numbers. In all cases they would serve the purpose of allowing users to add an additional layer of differentiation to these comments, especially to groups. It would still cover the use-case for colours, without linking them to visual state.

[jmchilton]

Variants would be better in my mind than color codes. I was thinking about something like variants yesterday... I don't think the bootstrap variants quite make sense so I was wondering if categorizing parts of a workflow for stylistic changes made sense - qa, preprocessing, postprocessing, core, visualization, ... but that seems impossible to do a priori. We could let use tag steps for tags that make semantic sense to them and then just do colors the way we do all the other tags... but I assume you'd not like that?

@mvdbeek do you share my particular concerns about colors? The concern with colors is less of representation thing - since that makes perfect sense and more of a long term Galaxy style editor concern.

[mvdbeek]

I do like the tag idea, and for screenshot purposes one could set a color that isn't necessarily persisted, or that is only optionally applied

[dannon]

+1 to tags, whether with user definable colors or not; this was what we discussed in a somewhat recent UI/UX meeting after one of the user discussions, right?

[ElectronicBlueberry]

The idea I had for variants would be to attach no inherent meaning to them, other than them being different from each other. E.g. two groups with variant 1, are different from a group with variant 2. This way the colour pallet can change completely, or be replaced by something different entirely. Choosing the meaning behind these variants is then (as it is with colours) up to the user.

I'd like to avoid giving the variants pre chosen purpose, as we can not predict everyones use case. Even if we cover most common use-cases for variants now, there is no guarantee they will remain the most common ones in the future.

I do not like the tag idea, or user definable colours. Both can lead to unpredictable and/or inconsistent results. Even if we fine-tune the colour generator to perfection, the chance is not small for two tag-like-coloured groups with different names to be assigned a visually very similar colour, negating their advantage. User definable colours would either need to be stored, in which case those would be stuck and may look bad in future workflow editor versions, or be ephemeral which would defeat most of their purpose.

[dannon]

I'm not sure I understand the distinction we're making between variants and tags. Tags wouldn't have inherent meaning or color, either, right? But, if desired, you definitely could embed tag meaning (whether in the name explicitly or as an annotation of some sort) and color information.

Obviously with some interface here, and these could be shown or ignored, but you could imagine some data like this:

tags:
    preprocessing:
        annotation: This tag indicates a step blah blah blah
        color: #11CC11
    mapping:
        annotation: This is the mapping process where we....
        color: #3333DD
    nonsense:
        annotation: This tag has no meaning.  But I'm using it anyway.  It also has no color specified.

[ElectronicBlueberry]

If we provide e.g. 10 variants, we can specify a colour for each, making sure those colours have good distinction to the editor and to each other. Tags would either need to automatically generate these colours, or store hex codes such as in your example, which leads to the issues outlined in my previous comment. A third option could be to combine these two, having users only choose from a preset pallet, which we could change in the future. That would however attach additional information to the workflow, and introduce much complexity in the UI, while having a near identical result to variants from a users perspective.

The data would look something like this:

groups:
     - title: some group
       variant: 0
     - title: some other group
       variant: 0
     - title: group doing something very different
       variant: 5

This is very simmilar to how colours currently work in this PR, with the semantic implication of "colour" removed.

[dannon]

I see where we're not connecting -- so I wasn't expecting we'd use the same tag-hash-to-color representation by default for tags; that is too prone to color collisions based on the name, etc. What I had in mind was the color selection scheme we use for workflow parameters -- we pick colors offset equidistant from one another. So it doesn't matter how many variants you have, they're always maximally far apart in color space (by default), though I'd imagine allowing users to specify their own specific colors would be desired.

Edit: To update the thread, this was discussed during the UI/UX meeting this week and we're going to stick with color handling as they are here. It's simpler and less likely to have issues than either programmatically choosing colorings or allowing specific user color inputs.

[AnnaSyme]

This looks great! Would it be possible to get a screenshot of the whole workflow and its annotations (such as colours around sections) as an item that can be put into the workflow report? That may be more a question for workflow reports rather than here.
Anyway thanks for doing this, it will be so useful for quickly understanding complicated workflows.

[dannon]

This is fantastic, and in a great spot to get in and in front of more folks. Thanks @ElectronicBlueberry!