Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ArgoCD OutOfSync if SeldonDeployment includes mountpoint #2811

Closed
RafalSkolasinski opened this issue Jan 4, 2021 · 3 comments · Fixed by #2852
Closed

ArgoCD OutOfSync if SeldonDeployment includes mountpoint #2811

RafalSkolasinski opened this issue Jan 4, 2021 · 3 comments · Fixed by #2852
Labels
Projects

Comments

@RafalSkolasinski
Copy link
Contributor

Describe the bug

In GitOps with ArgoCD setup if SeldonDeployment contains mountpoint argo reports incorrect OutOfSync error.
This goes away if one include

  ignoreDifferences:
  - group: machinelearning.seldon.io
    kind: SeldonDeployment
    jsonPointers:
    - /spec/predictors/0/componentSpecs/0/spec/containers/0/volumeMounts

in the Application definition.

See argocd diffing customisation.

To reproduce

  1. Install SC and ArgoCD
  2. Create following PVC
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test-pvc
  namespace: gitops-namespace
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
  1. Add following deployment via gitops
apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
  name: sklearn
spec:
  name: iris
  predictors:
  - componentSpecs:
    - spec:
        containers:
        - name: classifier
          resources:
            requests:
              memory: 50Mi
          volumeMounts:
          - name: pvc-storage
            mountPath: /new-pvc-location
        volumes:
        - name: pvc-storage
          persistentVolumeClaim:
            claimName: test-pvc

    graph:
      implementation: SKLEARN_SERVER
      modelUri: gs://seldon-models/sklearn/iris
      name: classifier
    name: default
    replicas: 1

Observe incorrect OutOfSync state.

Expected behaviour

ArgoCD does not report diff.

Environment

SC 1.5.x with ArgoCD 1.7.x or 1.8.x.

Model Details

image

@RafalSkolasinski RafalSkolasinski added bug triage Needs to be triaged and prioritised accordingly labels Jan 4, 2021
@ukclivecox ukclivecox added this to Triage in Backlog Jan 7, 2021
@ukclivecox ukclivecox removed the triage Needs to be triaged and prioritised accordingly label Jan 14, 2021
@ukclivecox ukclivecox moved this from Triage to Backlog in Backlog Jan 14, 2021
@RafalSkolasinski
Copy link
Contributor Author

Temporary workaround is to add

  ignoreDifferences:
  - group: machinelearning.seldon.io
    kind: SeldonDeployment
    jsonPointers:
    - /spec/predictors/0/componentSpecs/0/spec/containers/0/volumeMounts

to the argo application resource as per argocd documentation.

Problem will be solved by resolving #2817.

@omerfsen
Copy link
Contributor

+1

@omerfsen
Copy link
Contributor

Thank you for fixing #2817 as it will clear all "OutOfSnyc" Errors on ArgoCD even if I had a long list of ignoreDifferences

@ukclivecox ukclivecox added this to To do in Sprint 1 via automation Jan 21, 2021
@ukclivecox ukclivecox removed this from Backlog in Backlog Jan 21, 2021
@ukclivecox ukclivecox moved this from To do to Review in progress in Sprint 1 Jan 21, 2021
Sprint 1 automation moved this from Review in progress to Done Jan 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

3 participants