[DSIP-29][Task] Deprecate DISPATCH status in task instance lifecycle

[ruanwenjun]

Search before asking

• I had searched in the DSIP and found no similar DSIP.

Motivation

Once a TaskInstance is triggered, the status will be changed follow: submit -> dispatch -> running.
Submit means the task instance is initialized, dispatch means the task has been dispatch to worker, running means the task is running in worker.

The dispatch status is useful in the history design, since dispatch is a async operation, but in the latest implementation, the dispatch operation is a sync operation, maintain a dispatch will bring some problems.

1. When the task is in dispatch status, we cannot track it, we don't know whether the task is died since it done's initialize task instance log, we even donot know when the task will be executed.
2. When we want to kill a task instance in worker, it's hard to do kill operation, since the task may in waiting queue or in thread pool queue.

If we deprecated the dispatch status, things will be easy, once a task has dispatch to worker we can directly set the task instance status to running. This means we will also deprecate the waiting queue in worker, since the waiting queue doesn't help to manage the task, it just make the task lifecycle more difficult and make the resource management difficult, we hope the waiting task is all manage by master, once the task has been sent to worker, it will get thread to execute.

Design Detail

1. Remove the Dispatch status, once the task has been sent to worker then set task instance status to running.

3. Deprecate the TaskExecuteThreadsFullPolicy in worker, the worker will reject the task once the worker thread pool is full.

Compatibility, Deprecation, and Migration Plan

This is compatibility, since the dispatch status will keep in code but only set it is depracated, will remove this after 3.3.0.

Test Plan

Tested by UT and E2E

Code of Conduct

• I agree to follow this project's Code of Conduct