CancelActivity in Alteration API Fails to Update Activity State to Cancelled

[sfmskywalker]

When attempting to cancel an activity using the Alteration API, the API returns a successful message, indicating that the CancelActivity operation succeeded. However, upon inspecting the workflow instance, the state of the activity remains Running instead of updating to the expected Cancelled state.

Steps to Reproduce

1. Start a workflow containing an activity with its state set to Suspended.
2. Use the Alteration API to cancel this activity by sending the following payload:

   {
     "alterations": [
       {
         "type": "CancelActivity",
         "activityInstanceId": "87d5edfb7f30a1c5"
       }
     ],
     "workflowInstanceIds": ["3504d24c287e902a"]
   }

3. The API returns a success message, indicating the cancellation was processed:

   {
     "results": [
       {
         "workflowInstanceId": "3504d24c287e902a",
         "log": {
           "logEntries": [
             {
               "message": "CancelActivity succeeded",
               "logLevel": "Information",
               "timestamp": "2024-03-19T08:57:28.2839419+00:00"
             }
           ]
         },
         "workflowHasScheduledWork": false,
         "isSuccessful": true
       }
     ]
   }

4. Upon checking the workflow instance, the activity's state is incorrectly listed as Running instead of Cancelled.

Expected Behavior

After calling CancelActivity via the Alteration API, the targeted activity's state should be updated to Cancelled.

Actual Behavior

The activity's state remains Running despite the successful execution of a CancelActivity operation through the Alteration API.

This issue leads to inconsistencies within the workflow's state management and can affect the execution flow and the overall reliability of the workflow management system.

---

Note: It is crucial to ensure that state transitions through the Alteration API are accurately reflected within workflow instances to maintain the integrity and accuracy of workflow state tracking.

[sfmskywalker]

Issue Summary:

1. Cancellation Handling:

   • When a workflow instance is cancelled, the cancellation is properly enacted, but there are shortcomings in how the system captures and persists this state. Specifically, the system fails to retain a complete final state of the altered workflow instance.

2. State Extraction and Persistence Flaws:

   • The process of converting a workflow instance into a WorkflowExecutionContext and then back to a WorkflowInstance model does not account for cancelled ActivityExecutionContexts. This omission leads to the system not recognizing cancelled activities as part of the final state.
   • Moreover, new execution log records are not persisted by the AlterationRunner, which is critical for maintaining a comprehensive execution history.

3. Pipeline Execution and Middleware Issues:

   • Attempts to manually apply alterations outside the middleware pipeline were ineffective because the PersistBookmarkMiddleware component would not detect bookmark removal due to the pre-cancellation removal of bookmarks.

4. Proposed Solutions:

   • To address these issues, a new approach was proposed that involves creating a RunAlterationMiddleware, which would serve as the terminal component of the middleware pipeline, ensuring that all necessary states and records are processed and persisted.
   • A PR was created to clone the workflow execution pipeline and replace the terminal component with RunAlterationMiddleware. This change aligns the alterations process with the current pipeline setup, even when custom configurations are used during application bootstrap.

Result of Changes:

• With the new changes, cancelling activities now correctly updates their execution records to the Canceled state.
• Associated bookmarks are also appropriately deleted, resolving any route ambiguities that arise from the creation of bookmarks during the workflow trigger.
• Workflow execution log records are now preserved, ensuring accurate recording of actions in the journal.