Skip to content

Commit

Permalink
fix: add timeout for executor signal
Browse files Browse the repository at this point in the history 
Signed-off-by: Yuping Fan <coderfyp@gmail.com>
  • Loading branch information
Yuping Fan committed May 9, 2024
1 parent 6182386 commit bc7525d
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 2 deletions.
5 changes: 3 additions & 2 deletions docs/environment-variables.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ This document outlines environment variables that can be used to customize behav
| `ALL_POD_CHANGES_SIGNIFICANT` | `bool` | `false` | Whether to consider all pod changes as significant during pod reconciliation. |
| `ALWAYS_OFFLOAD_NODE_STATUS` | `bool` | `false` | Whether to always offload the node status. |
| `ARCHIVED_WORKFLOW_GC_PERIOD` | `time.Duration` | `24h` | The periodicity for GC of archived workflows. |
| `ARGO_PPROF` | `bool` | `false` | Enable [`pprof`](https://go.dev/blog/pprof) endpoints |
| `ARGO_PPROF` | `bool` | `false` | Enable [`pprof`](https://go.dev/blog/pprof) endpoints |
| `ARGO_PROGRESS_PATCH_TICK_DURATION` | `time.Duration` | `1m` | How often self reported progress is patched into the pod annotations which means how long it takes until the controller picks up the progress change. Set to 0 to disable self reporting progress. |
| `ARGO_PROGRESS_FILE_TICK_DURATION` | `time.Duration` | `3s` | How often the progress file is read by the executor. Set to 0 to disable self reporting progress. |
| `ARGO_REMOVE_PVC_PROTECTION_FINALIZER` | `bool` | `true` | Remove the `kubernetes.io/pvc-protection` finalizer from persistent volume claims (PVC) after marking PVCs created for the workflow for deletion, so deleted is not blocked until the pods are deleted. [#6629](https://github.com/argoproj/argo-workflows/issues/6629) |
Expand All @@ -31,6 +31,7 @@ This document outlines environment variables that can be used to customize behav
| `DISABLE_MAX_RECURSION` | `bool` | `false` | Set to true to disable the recursion preventer, which will stop a workflow running which has called into a child template 100 times |
| `EXPRESSION_TEMPLATES` | `bool` | `true` | Escape hatch to disable expression templates. |
| `EVENT_AGGREGATION_WITH_ANNOTATIONS` | `bool` | `false` | Whether event annotations will be used when aggregating events. |
| `EXECUTOR_SIGNAL_TIMEOUT` | `time.Duration` | `2m` | How long to wait after the executor has been terminated before timing out. |
| `GZIP_IMPLEMENTATION` | `string` | `PGZip` | The implementation of compression/decompression. Currently only "`PGZip`" and "`GZip`" are supported. |
| `INFORMER_WRITE_BACK` | `bool` | `true` | Whether to write back to informer instead of catching up. |
| `HEALTHZ_AGE` | `time.Duration` | `5m` | How old a un-reconciled workflow is to report unhealthy. |
Expand All @@ -53,7 +54,7 @@ This document outlines environment variables that can be used to customize behav
| `WF_DEL_PROPAGATION_POLICY` | `string` | `""` | The deletion propagation policy for workflows. |
| `WORKFLOW_GC_PERIOD` | `time.Duration` | `5m` | The periodicity for GC of workflows. |
| `SEMAPHORE_NOTIFY_DELAY` | `time.Duration` | `1s` | Tuning Delay when notifying semaphore waiters about availability in the semaphore |
| `WATCH_CONTROLLER_SEMAPHORE_CONFIGMAPS` | `bool` | `true` | Whether to watch the Controller's ConfigMap and semaphore ConfigMaps for run-time changes. When disabled, the Controller will only read these ConfigMaps once and will have to be manually restarted to pick up new changes. |
| `WATCH_CONTROLLER_SEMAPHORE_CONFIGMAPS` | `bool` | `true` | Whether to watch the Controller's ConfigMap and semaphore ConfigMaps for run-time changes. When disabled, the Controller will only read these ConfigMaps once and will have to be manually restarted to pick up new changes. |

CLI parameters of the Controller can be specified as environment variables with the `ARGO_` prefix.
For example:
Expand Down
6 changes: 6 additions & 0 deletions workflow/signal/signal.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ import (
"path/filepath"
"strings"
"syscall"
"time"

"github.com/argoproj/argo-workflows/v3/util/env"

log "github.com/sirupsen/logrus"
corev1 "k8s.io/api/core/v1"
Expand Down Expand Up @@ -49,6 +52,9 @@ func ExecPodContainerAndGetOutput(ctx context.Context, restConfig *rest.Config,
if err != nil {
return err
}
executorSignalTimeout := env.LookupEnvDurationOr("EXECUTOR_SIGNAL_TIMEOUT", 2*time.Minute)
ctx, cancel := context.WithTimeout(ctx, executorSignalTimeout)
defer cancel()
stdout, stderr, err := common.GetExecutorOutput(ctx, x)
log.
WithField("namespace", namespace).
Expand Down

0 comments on commit bc7525d

Please sign in to comment.