Enable Workflows to send a success/fail response to users

[qqmyers]

Currently, when workflows complete, they may succeed/fail. However, because they run asynchronously, there has been no mechanism for them to communicate with users and the only record of success/failure was in log messages and, for pre-publication workflows, the failure to finalize publication (which results in the Dataset going back to being in the draft state). For prepublication workflows in particular and workflows in general, it would be useful for them to be able to send success and failure messages to users. Use cases include:

• prepublication workflows adding metadata (e.g. DANS)
• prepublication spam detection workflow (proposed) which would indicate why publication was not allowed
• postpublication archiving workflow - indicate success/failure in creating an archival copy

The best option for prepublication workflows, at least visually, seems to be adding a message to the dataset page (as with other success/fail messages). However, because workflows are asynchronous, and potentially long lived, and can be triggered by publishing via API, the user may not be viewing the dataset page. So, it looks like sending a UserNotification (which appears in their notification tab and triggers an email) seems to be needed for a general mechanism. It may be possible to also display a banner-style message in addition/instead (instead would only be a good option if there's a reliable way to tell if the user is going to see a banner).

Sending a notification should be relatively straight forward, so I'd suggest doing that as an initial step and then, if there's interest, looking into whether workflows can keep track of the view state well enough to send a banner-style message. (Any ideas on how to handle the latter would be appreciated!).

Some things to consider:

• how to manage messages from multiple steps (do steps interact? an initial choice might be to just have steps messaging be independent as most workflows only have one substantial step anyway (some use the log step before and after the main step).)
• whether success/fail messages are required or optional (and is the choice dynamic?)
• how external workflows using, for example, the http s/r step can convey message text
• the message for an external workflow that can't be contacted (e.g. the http call to the external service fails)

For the last, https/r currently uses a regexp to identify whether the step succeeded or failed. This mechanism is useful assuming legacy apps where the response cannot be specified. It could be extended by using the regexp to identify a subset of the response as the message. Alternately, that step, or a variant could expect a standard ~json format (object with 'status' and 'message').

Thoughts welcome. I'll plan to update the issue with more concrete proposal(s).

[janvanmansum]

It would be good to at least have generic message on the dataset details page that tells you that publication failed and maybe refers you to the notifications for the details. I would expect something similar to the banner that currently displays when the dataset is locked.