`cleanupSubscription` is not working

[freeznet]

Pulsar Functions Worker cleans the subscription in Functions Worker, see https://github.com/apache/pulsar/blob/branch-2.10/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionActioner.java#L326-L343

Function Mesh only passes the cleanupSubscription to the function details, but without function worker, the config is not working on Function Mesh cluster.

[tpiperatgod]

Dose it mean we need to add a mechanism to inform the Function Woker to exec terminateFunction when the user deletes functions in the Function Mesh cluster?

[freeznet]

@tpiperatgod not exactly, the better solution is to call pulsar admin apis to remove the subscription if the function is been deleted.

[jiangpengcheng]

Proposal - Using k8s Finalizers: https://book.kubebuilder.io/reference/using-finalizers.html

Option 1: Start a k8s job to delete subscription

1. when function/sink/source is created or updated and has set cleanupSubscription=true or is a batchSource, create a job that uses the same container with function/sink/source; the command is to delete subscription and topic(for batch source), and the suspend is true, and the TTLSecondsAfterFinished is 0

2. since the job requires the same config and secret used by function/sink/source, and these resources may be deleted too when function/sink/source is deleted, we need to clone all of the used config and secret with new names, such as add a suffix -cleanup-job to the original name and set the owner to the job

3. add a Finalizers to the function/sink/source

4. when function/sink/source is updated to set cleanupSubscription=false or is not a batchSource, delete the cleanup job, and delete the Finalizers from function/sink/source

5. if function/sink/source is deleting, set the suspend of job to false, and delete the Finalizers from function/sink/source

6. then function/sink/source is deleted, and the cleanup job will also finish the cleanup

The flow chart looks like below:

Option 2: Exec to function/sink/source pod directly to do clean up

1. when function/sink/source is created or updated and has set cleanupSubscription=true or is a batchSource, add a Finalizers to the function/sink/source

2. when function/sink/source is updated to set cleanupSubscription=false or is not a batchSource, delete the Finalizers from function/sink/source

3. if function/sink/source is deleting, if its replicas is 0, scale it to 1, and exec to one of the function container to do subscription*topic deletion job, and delete the Finalizers from function/sink/source

The flow chart looks like below: