Activity fails when resuming with bookmark

[msavencov]

Hi, i'm trying to create a blocking activity FormSubmit for integration with external app. All works fine but...
I have a issue with materialization of my activity after suspension.
Below is shown the faulted activity which cannot resolve the FormState input variable because of javascript function getOutputFrom("RSPSearch1", "Result") and getOutputFrom("RSPSearch2", "Result") returns null on resuming suspended activity.

workflow definition, faulted instance and custom activities are attached below

The FormSubmit2.FormState input has the following javascript code:

let first = getOutputFrom("RSPSearch1", "Result") // returns null on resuming and has valid value before suspension
let second = getOutputFrom("RSPSearch2", "Result") // returns null on resuming and has valid value before suspension

return [...first, ...second]

1. The activity FormSubmit2 successfully reaches the ExecuteAsync method and creates a FormBookmarkPayload: at this, the FormState input has a valid json array, concatenated from output of previous two activities.
2. The activity FormSubmit2 becomes suspended
3. Submitting an inbox message with FormBookmarkPayload bookmark

        // input is one of item from `FormSubmit.FormState` selected on `template2` form
        // var input = new {
        //     template2 = new {
        //         Identity = "2998816868032",
        //         FirstName = "John 372",
        //         LastName = "Doe139"
        //     }
        //  };
        var bookmark = new FormBookmarkPayload("form2", "template2");
        var options = new WorkflowInboxMessageDeliveryOptions
        {
            DispatchAsynchronously = true
        };
        var inboxMessage = NewWorkflowInboxMessage.For<FormSubmit>(bookmark, workflowInstanceId, correlationId, activityInstanceId, input);
        var result = await workflowInbox.SubmitAsync(inboxMessage, options, ct);

4. The activity FormSubmit2 fails before invoking FormSubmit.OnResume method with exception:

Jint.Runtime.JavaScriptException+JavaScriptErrorWrapperException, Jint: Cannot convert undefined or null to object
   at Jint.Engine.ScriptEvaluation(ScriptRecord scriptRecord)
   at Jint.Engine.<>c__DisplayClass92_0.<Execute>b__0()
   at Jint.Engine.ExecuteWithConstraints[T](Boolean strict, Func`1 callback)
   at Jint.Engine.Execute(Script script)
   at Jint.Engine.Evaluate(Script script)
   at Jint.Engine.Evaluate(String code, String source, ParserOptions parserOptions)
   at Jint.Engine.Evaluate(String code)
   at Elsa.JavaScript.Services.JintJavaScriptEvaluator.ExecuteExpressionAndGetResult(Engine engine, String expression)
   at Elsa.JavaScript.Services.JintJavaScriptEvaluator.EvaluateAsync(String expression, Type returnType, ExpressionExecutionContext context, ExpressionEvaluatorOptions options, Action`1 configureEngine, CancellationToken cancellationToken)
   at Elsa.JavaScript.Expressions.JavaScriptExpressionHandler.EvaluateAsync(Expression expression, Type returnType, ExpressionExecutionContext context, ExpressionEvaluatorOptions options)
   at Elsa.Expressions.Services.ExpressionEvaluator.EvaluateAsync(Expression expression, Type returnType, ExpressionExecutionContext context, ExpressionEvaluatorOptions options)
   at Elsa.Extensions.ActivityExecutionContextExtensions.EvaluateInputPropertyAsync(ActivityExecutionContext context, ActivityDescriptor activityDescriptor, InputDescriptor inputDescriptor)
   at Elsa.Extensions.ActivityExecutionContextExtensions.EvaluateInputPropertiesAsync(ActivityExecutionContext context)
   at Elsa.Workflows.Middleware.Activities.DefaultActivityInvokerMiddleware.EvaluateInputPropertiesAsync(ActivityExecutionContext context)
   at Elsa.Workflows.Middleware.Activities.DefaultActivityInvokerMiddleware.InvokeAsync(ActivityExecutionContext context)
   at Elsa.Workflows.Middleware.Activities.NotificationPublishingMiddleware.InvokeAsync(ActivityExecutionContext context)
   at Elsa.Workflows.Middleware.Activities.ExecutionLogMiddleware.InvokeAsync(ActivityExecutionContext context)
   at Elsa.Workflows.Middleware.Activities.ExceptionHandlingMiddleware.InvokeAsync(ActivityExecutionContext context)

===

FormSubmit activitity

[Activity("OIM", "Forms", "Wait for form submission", Kind = ActivityKind.Trigger)]
public class FormSubmit : Trigger
{
    [Input(Description = "The form identifier")]
    public Input<string> FormId { get; set; } = new(Guid.NewGuid().ToString());

    [Input(Description = "The form name")]
    public Input<string> FormName { get; set; } = default!;

    [Input(Description = "The form state")]
    public Input<object> FormState { get; set; } = default!;

    [Output(Description = "The collected form data")]
    public Output<object> FormData { get; set; } = null!;

    protected override object GetTriggerPayload(TriggerIndexingContext context)
    {
        var formId = FormId.Get(context.ExpressionExecutionContext);
        var formName = FormName.Get(context.ExpressionExecutionContext);

        return new FormBookmarkPayload(formId, formName);
    }

    protected override async ValueTask ExecuteAsync(ActivityExecutionContext context)
    {
        if (context.IsTriggerOfWorkflow())
        {
            await OnResume(context);
            return;
        }
        
        var formId = FormId.Get(context);
        var formName = FormName.Get(context);
        
        context.CreateBookmark(new FormBookmarkPayload(formId, formName), OnResume, false);
    }

    private async ValueTask OnResume(ActivityExecutionContext context)
    {
        var formName = FormName.Get(context);
        var formData = context.GetWorkflowInput<object>(formName);

        context.Set(FormData, formData);
        
        await context.CompleteActivityAsync();
    }
}

public record FormBookmarkPayload(string Id, string Name);

===

RSPSearch activitity

[Activity("OIM", "MConnect", "Find a person in R.S.P.")]
public class RSPSearch : Activity<PersonSearchResults>
{
    [Input] public Input<string> IDNP { get; set; } = default!;

    protected override async ValueTask ExecuteAsync(ActivityExecutionContext context)
    {
        var identity = IDNP.Get(context.ExpressionExecutionContext);
        var results = new PersonSearchResults
        {
            new PersonSearchResult
            {
                Identity = identity,
                FirstName = "John " + random.Next(100, 999),
                LastName = "Doe " + random.Next(100, 999)
            }
        };
        
        context.SetResult(results);

        await context.CompleteActivityAsync();
    }
}

public class PersonSearchResult
{
    public string Identity { get; set; } = null!;
    public string FirstName { get; set; } = null!;
    public string LastName { get; set; } = null!;
}
public class PersonSearchResults : List<PersonSearchResult>;

===

Workflow Definition

workflow-definition.json

===

Workflow Instance

workflow-instance.json

[glime-ai]

Based on the information provided, it seems that the issue you are encountering is related to the JavaScript expressions getOutputFrom("RSPSearch1", "Result") and getOutputFrom("RSPSearch2", "Result") returning null when the FormSubmit2 activity is resumed after being suspended. This behavior suggests that the outputs of the RSPSearch1 and RSPSearch2 activities are not being persisted or are not accessible upon resumption.

To address this issue, you may want to ensure that the outputs of the RSPSearch1 and RSPSearch2 activities are being correctly stored and that they are available in the workflow context when the FormSubmit2 activity is resumed. Here are a few steps you can take to troubleshoot and potentially resolve the issue:

1. Verify that the outputs of RSPSearch1 and RSPSearch2 are being set correctly before the workflow is suspended.
2. Check if the workflow context is being saved and loaded correctly upon suspension and resumption.
3. Ensure that the JavaScript functions getOutputFrom are implemented correctly and are capable of retrieving the outputs from the specified activities.
4. Review the workflow definition and instance JSON files to confirm that the outputs are present and correctly formatted.
5. If you are using Elsa version 2 or above, make sure you are following the updated way of defining and accessing activity inputs and outputs as per the documentation.

If the issue persists, you may want to reach out to the Elsa community or support channels with specific details about your implementation and the errors you are encountering. Additionally, you can review the Elsa documentation and resources for more information on defining and using activities, handling inputs and outputs, and managing workflow execution.

Please let me know if you need further assistance or if there are any specific areas you would like me to elaborate on.

[sfmskywalker]

The issue is that activity output is transient, which means that when the workflow enters the Suspended state, the output will be lost. When the workflow is then resumed, the Form Submit activity will fail as you reported, given that it is trying to get activity output that will result in null values being returned.

To fix this, you can capture the activities output using a persistent variable.
The Form Submit activity can then use those variables, which will have a copy of the output values.

Here's a workflow definition that demonstrates this:

modified-workflow-definition.json

This will fix the loss of output values.
However, I ran into another issue that your custom activities uncovered; upon attempting to deserialize the elements of the PersonSearchResults collection, which is a class that derives from List<PersonSearchResult>, the serializer fails to recognize that it should deserialize the individual elements as PersonSearchResult, and instead, deserializes them as ExpandoObjects.

I have pushed a fix to the main branch for this. If you want to take advantage of this fix, please update your Elsa package references to the latest 3.2.0 preview build (3.2.0-preview.1393 or higher).

[sfmskywalker]

Based on the information provided, it seems that the issue you are encountering is related to the JavaScript expressions getOutputFrom("RSPSearch1", "Result") and getOutputFrom("RSPSearch2", "Result") returning null when the FormSubmit2 activity is resumed after being suspended. This behavior suggests that the outputs of the RSPSearch1 and RSPSearch2 activities are not being persisted or are not accessible upon resumption.

The bot was on to something 😄

[sfmskywalker]

I will proceed to closing this issue, but feel free to continue the conversation.

[sfmskywalker]

More information about activity output & direct access can be found here: https://elsa-workflows.github.io/elsa-documentation/custom-activities.html#direct-access

[msavencov]

Hi @sfmskywalker
Thanks for the clarified answer and sorry for the late response (I've been very busy these past weeks).
At that time, I solved the issue in a similar way by setting a variable using the "Set Variable" activity in front of my "Submit Form".

Now I have another question: can I instruct the engine to always save the results of the desired activities (or all activities in my workflow) so that I can access them from anywhere in the workflow? And how to enable that if engine allows that.

Thanks, have a nice day!

[sfmskywalker]

Hi @msavencov , no worries, I am happy to hear you found a workaround 👍🏻
For now, the engine doesn't have a configurable way to store an activity's output, but do I have been thinking about adding this capability.

[gorillapower]

Glad I found this issue, I was tearing my hear out wondering why the outputs were not persisting. This would be a convienient feature for sure. For now I will look into setting global variables as a workaround.

Is this something worth creating a separate issue for?

[sfmskywalker]

@gorillapower That’s a good idea 👍