Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

calico-node health checks break after editing in the UI #22848

Closed
bentastic27 opened this issue Sep 13, 2019 · 4 comments
Closed

calico-node health checks break after editing in the UI #22848

bentastic27 opened this issue Sep 13, 2019 · 4 comments
Assignees
Labels
internal kind/bug Issues that are defects reported by users or that we know have reached a real release
Milestone

Comments

@bentastic27
Copy link
Contributor

What kind of request is this (question/bug/enhancement/feature request):
bug

Steps to reproduce (least amount of steps as possible):

create cluster using canal
add an environment variable to the calico-node deployment via the UI and hit save

Result:

The host field for the liveness/readiness checks for the calico-node containers will disappear to look like this:

        livenessProbe:
          failureThreshold: 6
          httpGet:
            path: /liveness
            port: 9099
            scheme: HTTP

This causes the checks to go to the nodes IP address over port 9099, which does not work, causing the pod to crashloop. Host was originally set to localhost.

Environment information

  • Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI):
    2.2.6

  • Installation option (single install/HA):
    either

Cluster information

  • Cluster type (Hosted/Infrastructure Provider/Custom/Imported): custom
  • Machine type (cloud/VM/metal) and specifications (CPU/memory): any
  • Kubernetes version (use kubectl version): 1.14.3
@bentastic27 bentastic27 added kind/bug Issues that are defects reported by users or that we know have reached a real release internal labels Sep 13, 2019
@superseb
Copy link
Contributor

This is a known issue, see #20749.

@loganhz
Copy link

loganhz commented Sep 13, 2019

Fixed in 2.3.0

@jiaqiluo
Copy link
Member

The bug is a duplication of #20749
see the mentioned issue for validation result.

@mattmattox
Copy link

::Workaround::

  • Go to Rancher UI
  • Browse to System Project -> kube-system Namespace -> canal Workload
  • Click the action menu and select "View/Edit as YAML"
  • Edit the livenessProbe
    Before:
    livenessProbe:
    failureThreshold: 6
    httpGet:
    path: /liveness
    port: 9099
    scheme: HTTP
    After:
    livenessProbe:
    failureThreshold: 6
    httpGet:
    host: localhost
    path: /liveness
    port: 9099
    scheme: HTTP
  • Edit the readinessProbe
    Before:
    readinessProbe:
    failureThreshold: 6
    httpGet:
    path: /readiness
    port: 9099
    scheme: HTTP
    After:
    readinessProbe:
    failureThreshold: 6
    httpGet:
    host: localhost
    path: /readiness
    port: 9099
    scheme: HTTP
  • Click Save
  • Wait a few mins for all canal pods to go return to healthy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
internal kind/bug Issues that are defects reported by users or that we know have reached a real release
Projects
None yet
Development

No branches or pull requests

7 participants