EG does not cleanup Envoy proxy deployments if a gateway delete notification is missed

[alexwo]

Description:
The issue observed is that orphaned Envoy proxy deployments are not removed when a gateway delete notification is missed.

It is expected that during a full reconcile, EG can identify and clean up these orphaned proxies to maintain consistency and prevent resource leakage. finalizers are missing.

Repro steps:
1. Create a gateway and observe that the Envoy proxy deployment is created successfully.
2. Scale EG to 0 and delete the gateway resource
3. Scale EG back to 1 to force a full reconcile
4. Observe that the orphaned Envoy proxy deployment is not removed.

[arkodg]

this can be solved with #3051

[Skaronator]

Same with PDB and HPA resources. They won't get deleted when removing a Gateway or disabling them in the EnvoyProxy config

[arkodg]

#3051 cannot be implemented because ownerReferences dont have cross namespace support

[MaYuan-02]

Same with PDB and HPA resources. They won't get deleted when removing a Gateway or disabling them in the EnvoyProxy config

@arkodg I'm also encountering issues with deleting HPA and PDB resources. Is anyone currently working on resolving this?

[arkodg]

hey @MaYuan-02 are you hitting #4380 which is specific to custom named resources ?

this issue is around clean up when the EG controller does not exist, so there is no one around to clean up the generated resources

[arkodg]

reg this issue, we may need to implement mark and sweep in the infra init function, so it cleans up orphaned resources that once belonged to it

[MaYuan-02]

@arkodg Thanks for your reply.
I'm facing an issue where turning off HPA in the EnvoyProxy resource doesn't remove the HPA resource, leading to mismatches with the gateway's replica count. The control plane only deletes resources when it receives an InfraIR delete message, which disabling HPA doesn't trigger. I've opened a new issue(#5357) to track this.

[MaYuan-02]

reg this issue, we may need to implement mark and sweep in the infra init function, so it cleans up orphaned resources that once belonged to it

Yes, we could add logic to clean up orphaned resources in the initInfra after the leader election. Is anyone currently working on fixing this issue? If not, I'd be happy to give it a try.