Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Istio-Pilot / Istiod HPA shows 0% cpu utilization #24809

Closed
Prashansa-K opened this issue Jun 19, 2020 · 4 comments · Fixed by #24887
Closed

Istio-Pilot / Istiod HPA shows 0% cpu utilization #24809

Prashansa-K opened this issue Jun 19, 2020 · 4 comments · Fixed by #24887
Assignees
Labels

Comments

@Prashansa-K
Copy link

Prashansa-K commented Jun 19, 2020

Bug description
On installing istio, an HPA is created named as 'istiod' and target deployment as 'istio-pilot' which shows 0% cpu utilization at all times. In 1.6.0, there is no specific deployment called istio-pilot. Even if I edit the HPA and put istiod as scaleTargetRef Deployment, then too it shows 0%.

[X] Configuration Infrastructure
[ ] Docs
[X] Installation
[ ] Networking
[X] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

Expected behavior
HPA should be configured based on proper deployment and show correct cpu utilization.

Steps to reproduce the bug

Version (include the output of istioctl version --remote and kubectl version and helm version if you used Helm)
1.6.0

How was Istio installed?
istioctl

Environment where bug was observed (cloud vendor, OS, etc)
AWS EKS

@howardjohn
Copy link
Member

Where does that HPA come from?
~/istio/istio-1.6.0/bin/istioctl manifest generate shows:

# Cni component is disabled.

# EgressGateways istio-egressgateway component is disabled.

# Resources for IngressGateways component

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  labels:
    app: istio-ingressgateway
    istio: ingressgateway
    release: istio
  name: istio-ingressgateway
  namespace: istio-system
spec:
  maxReplicas: 5
  metrics:
  - resource:
      name: cpu
      targetAverageUtilization: 80
    type: Resource
  minReplicas: 1
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: istio-ingressgateway

---

# IstiodRemote component is disabled.

# Resources for Pilot component

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: istiod
  namespace: istio-system
  labels:
    app: istiod
    release: istio
    istio.io/rev: default
spec:
  maxReplicas: 5
  minReplicas: 1
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: istiod
  metrics:
  - type: Resource
    resource:
      name: cpu
      targetAverageUtilization: 80

No istio-pilot is found

@howardjohn howardjohn added the kind/need more info Need more info or followup from the issue reporter label Jun 19, 2020
@MarkDeckert
Copy link

MarkDeckert commented Jun 22, 2020

I have the same issue. 1.6.2. I have a custom profile slightly modified from a dump of the default profile. I didn't modify anything istiod or pilot related.

But check out this clean example even without any mods:

istiod hpa has correct target label (scaleTargetRef.name: istiod):

istioctl manifest generate --set profile=default |kubectl apply -f -

istio hpa has wrong target label (scaleTargetRef.name: istio-pilot):

istioctl profile dump default > profile.yaml
istioctl manifest generate -f profile.yaml |kubectl apply -f -

zero modifications of the profile.yaml file

@howardjohn
Copy link
Member

@richardwxn ^

@richardwxn
Copy link
Contributor

taking a look

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants