Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support sub-workflow dependency handling of JSON workflows #4098

Merged
merged 11 commits into from
Jun 3, 2023

Conversation

sfmskywalker
Copy link
Member

@sfmskywalker sfmskywalker commented Jun 3, 2023

BREAKING CHANGE

This PR adds a new column called ProviderName to the WorkflowDefinition entity.
Migrations of the Management module have been replaced that include this column.

When you maintain your own migrations, you should generate an "update" step, or recreate them if you don't mind deleting your database first.

Details

This PR updates the workflow provider system with support for handling workflows that use other workflows as activities.

Before this PR, there would be an issue if you had e.g. two workflow files called Main and Sub, where Sub is configured as "usable as activity" and Main has an activity reference to Sub.

Problem 1: when the system populates the activity registry, it would not see the Sub workflow, since it isn't part of the workflow store at that time.
Problem 2: when the system populates the workflow store from JSON files, and the Main workflow would be encountered before Sub, the Main workflow's activity reference to Sub would be replaced with a NotFoundActivity.

To solve this, the workflows from providers are loaded twice, using the following steps:

  1. Populate activity registry
  2. Populate workflow store from providers (e.g. JSON files)
  3. Populate activity registry again - this time it finds workflows usable as activity, e.g. the Sub workflow
  4. Populate workflow store from providers again (e.g. JSON files) - this time, the Main workflow will find the Sub workflow available as an activity.

One potential concern is that the larger the number of workflows, the longer it takes for the application to startup.

As a future optimisation, we could provide "discovery" data while loading e.g. JSON files to see if any yielded workflows contain NotFoundActivity references from certain providers, and reload only these providers and update the activity registry for just those providers.

Note that workflows created in the DB directly (e.g. via the workflow designer) are not affected by this, since they already exist there and there is no workflow provider that reads from this database (the point of workflow providers is to discover workflows from other sources and store them in the DB).

cc @MariusVuscanNx @cristinamudura

@sfmskywalker sfmskywalker merged commit 0bb2ddc into v3 Jun 3, 2023
@sfmskywalker sfmskywalker deleted the v3-json-provider-dependencies branch June 3, 2023 10:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant