enable default podGC for workflowtemplate reference scenario

[catweisun]

Summary

Need centralized podGC to support workflow template scenario.

Motivation

I'm trying to evaluate and optimize the performance for the worklfow on argo, but the default podGC strategy is not support the workflowtemplate ref scenario and I have to setup the podGC in individual workflowtemplate as workaround. The workflow template is developed by other vendors/partners who focus on the business logic instead infrastructure and optimization, it make no sense to add podGC spec

Please give examples of your use case, e.g. when would you use this.

Proposal

Enable the default podGC in controller configMap for workflow template should be best and easy way for this scenario.

---

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

[jacobdr]

We used to enable workflowDefaults in the ConfigMap to be:

workflowDefaults: |
    podGC:
        strategy: OnWorkflowSuccess

This worked in the 2.8.x release train, but broke in 2.9.1

In 2.9.1, when submitting a job through argo submit via the CLI, an exception is thrown. So we had to roll that back and just add the podGC to the submitted template. I think this ticket covers this so I won't open another issue. Wanted to mention it as another case that might help with reproduction

[sarabala1979]

I am working on it

[alexec]

@jacobdr I think you are raising a bug with v2.9 rather than comment on an enhancement for v2.10. Can you please open another issue for you bug?