DAG doesn't consider retryStrategy.backoff.maxDuration when determining if a Retry node has exhausted its retries

[simster7]

If a Retry node is not complete on a DAG, assessDAGPhase considers if the node has more retries available. If it does, it allows the DAG to continue Running.

https://github.com/argoproj/argo/blob/3c4422326dceea456df94a71270df80e9cbf7177/workflow/controller/dag.go#L141-L145

However, hasMoreRetries does not consider retryStrategy.backoff.maxDuration in this assessment:

https://github.com/argoproj/argo/blob/3c4422326dceea456df94a71270df80e9cbf7177/workflow/controller/dag.go#L225-L227

This can dead-lock the controller:

1. A retry node exhausts its maxDuration (but not its limit). This prevents new nodes from being created.
2. Operator arrives at assessDAGPhase. Since limit hasn't been exceeded, the DAG (and therefore the Workflow) is marked as running.
3. Workflow remains running, back to step 1.