Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Argo controller had rolled out negative values during the promotion #3589

Closed
2 tasks
sathishmr-zepto opened this issue May 22, 2024 · 1 comment
Closed
2 tasks
Labels
bug Something isn't working

Comments

@sathishmr-zepto
Copy link

Checklist:

  • I've included steps to reproduce the bug.
  • I've included the version of argo rollouts.

Describe the bug

the Argo rollout logs, we discovered that the Argo controller had configured negative values during the canary promotion, consequently triggering the issue.

To Reproduce

Expected behavior

Screenshots

Screenshot 2024-05-21 at 4 23 21 PM

Version
1.60+7eae71e
Screenshot 2024-05-22 at 4 30 06 PM

Logs

# Paste the logs from the rollout controller

# Logs for the entire controller:
kubectl logs -n argo-rollouts deployment/argo-rollouts

2024-05-16 15:15:57.018 | time="2024-05-16T09:45:57Z" level=info msg="Patched: {\"status\":{\"canary\":{\"weights\":{\"canary\":{\"weight\":-25},\"stable\":{\"weight\":125}}}}}" generation=1347 namespace=search-platform resourceVersion=1688392998 rollout=search-service |  
-- | -- | --
  |   | 2024-05-16 15:15:56.984 | time="2024-05-16T09:45:56Z" level=info msg="Event(v1.ObjectReference{Kind:\"Rollout\", Namespace:\"search-platform\", Name:\"search-service\", UID:\"b3856f40-a846-4d04-9d85-a17b7f6c4676\", APIVersion:\"argoproj.io/v1alpha1\", ResourceVersion:\"1688392998\", FieldPath:\"\"}): type: 'Normal' reason: 'TrafficWeightUpdated' Traffic weight updated from 0 to -25" |  
  |   | 2024-05-16 15:15:56.984 | time="2024-05-16T09:45:56Z" level=info msg="Traffic weight updated from 0 to -25" event_reason=TrafficWeightUpdated namespace=search-platform rollout=search-service |  
  |   | 2024-05-16 15:15:56.984 | time="2024-05-16T09:45:56Z" level=info msg="New weights: &TrafficWeights{Canary:WeightDestination{Weight:-25,ServiceName:,PodTemplateHash:fcc6d7569,},Stable:WeightDestination{Weight:125,ServiceName:,PodTemplateHash:58db58969f,},Additional:[]WeightDestination{},Verified:nil,}" namespace=search-platform rollout=search-service |  
  |   | 2024-05-16 15:15:56.984 | time="2024-05-16T09:45:56Z" level=info msg="Previous weights: &TrafficWeights{Canary:WeightDestination{Weight:0,ServiceName:,PodTemplateHash:fcc6d7569,},Stable:WeightDestination{Weight:100,ServiceName:,PodTemplateHash:58db58969f,},Additional:[]WeightDestination{},Verified:nil,}" namespace=search-platform rollout=search-service |  
  |   | 2024-05-16 15:13:36.229 | time="2024-05-16T09:43:36Z" level=info msg="Patched: {\"status\":{\"canary\":{\"weights\":{\"canary\":{\"weight\":60},\"stable\":{\"weight\":40}}},\"conditions\":[{\"lastTransitionTime\":\"2024-05-15T12:27:31Z\",\"lastUpdateTime\":\"2024-05-15T12:27:31Z\",\"message\":\"Rollout has minimum availability\",\"reason\":\"AvailableReason\",\"status\":\"True\",\"type\":\"Available\"},{\"lastTransitionTime\":\"2024-05-16T09:37:51Z\",\"lastUpdateTime\":\"2024-05-16T09:37:51Z\",\"message\":\"Rollout is not healthy\",\"reason\":\"RolloutHealthy\",\"status\":\"False\",\"type\":\"Healthy\"},{\"lastTransitionTime\":\"2024-05-16T09:37:51Z\",\"lastUpdateTime\":\"2024-05-16T09:37:51Z\",\"message\":\"RolloutCompleted\",\"reason\":\"RolloutCompleted\",\"status\":\"False\",\"type\":\"Completed\"},{\"lastTransitionTime\":\"2024-05-16T09:43:36Z\",\"lastUpdateTime\":\"2024-05-16T09:43:36Z\",\"message\":\"Rollout is paused\",\"reason\":\"RolloutPaused\",\"status\":\"False\",\"type\":\"Paused\"},{\"lastTransitionTime\":\"2024-05-16T09:43:36Z\",\"lastUpdateTime\":\"2024-05-16T09:43:36Z\",\"message\":\"ReplicaSet \\\"wms-outbound-86448677c\\\" is progressing.\",\"reason\":\"ReplicaSetUpdated\",\"status\":\"True\",\"type\":\"Progressing\"}],\"currentStepIndex\":17}}" generation=1000 namespace=supply-chain resourceVersion=1688383615 rollout=wms-outbound


# Logs for a specific rollout:
kubectl logs -n argo-rollouts deployment/argo-rollouts | grep rollout=<ROLLOUTNAME

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritize the issues with the most 👍.

@sathishmr-zepto sathishmr-zepto added the bug Something isn't working label May 22, 2024
@zachaller
Copy link
Collaborator

zachaller commented Jun 5, 2024

Closed via #3474

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants