Do not block template creation, when a task is still being executed (blocking parallel execution)

[ViniciusSouza]

Today, when a template with a long running action is being executed it blocks new templates creation.

We have developed an integration with Argo Workflow and some workflows take some time to complete, in order to be able to use the output of the workflow on the next step. We have a wait-workflow execution action, that keeps querying the Ago Status API, when the workflow completed (success or not) we continue the steps.

As you can see at the picture above, we are not able to execute others template until the process finishes.

Feature Suggestion

Using worker threads to create to execute the action steps.

Possible Implementation

Scaffolder tasks, more specifically the TaskWorker.ts can create a new thread to avoid a long running task to block others tasks.

Context

I put together this repository to help illustrate the issue.

[Rugvip]

This is down to the number of task worker, which is configurable through taskWorkers when creating the scaffolder backend router.

backstage/plugins/scaffolder-backend/src/service/router.ts

Line 64 in 3da5211

taskWorkers?: number;

The default of a single worker seems a bit silly to me though, I think we can dial that up a bit to make the default experience a bit nicer. Perhaps at least 3 workers by default? @benjdlambert any thoughts or ongoing work here? I know we want to make the broker better at being able to distribute tasks among nodes too

[benjdlambert]

Yep, looks like we could probably increase the amount by default to 3 or so. We've also discussed being able to schedule these tasks on with Kubernetes Jobs, but nothing has been planned yet. This could give us a little more freedom to maybe remove some of these TaskWorker limits, and just have the scheduler push jobs to K8s or some other scheduling interface. But nothing solid yet. @ViniciusSouza, do you want to contribute upping the default limit? Also in the meantime you can set taskWorkers: 3 as an option to the createRouter function in the packags/backend/src/plugins/scaffolder.ts to work around this issue.