Feat: need one new Trait to support HorizontalPodAutoscaler of CPU/ME… #5225

StevenLeiZhang · 2022-12-24T04:00:27Z

…M/PodCustomMetrcs

Signed-off-by: StevenLeiZhang zhangleiic@163.com

Description of your changes

I have:

Read and followed KubeVela's contribution process.
Related Docs updated properly. In a new feature or configuration option, an update to the documentation is necessary.
Run make reviewable to ensure this PR is ready for review.
Added backport release-x.y labels to auto-backport this PR if necessary.

How has this code been tested

CLI Version: v1.6.5
Core Version: 
GitRevision: git-18639cc
GolangVersion: go1.19.4

intall velaUX and follow https://github.com/kubernetes-sigs/metrics-server/blob/master/README.md to install metric-server on my MacBook
apply this Trait

ZBMac-C02CJ6QKM:trait$ vela def apply hpa.cue
TraitDefinition hpa created in namespace vela-system.
ZBMac-C02CJ6QKM:trait $

3.go to VelaUX,Definitions->TraitDefinition->hpa, to view its details
check the uiSchema can be generated normally and parameters can be show on the panel

4.apply one Application with this Trait

cd vela-templates/definitions/usage-examples
 kubectl apply -f application-with-hpa.yaml

check the Applicaions can be up

ZBMac-C02CJ6QKM:~ $ vela ls
APP             COMPONENT       TYPE            TRAITS          PHASE   HEALTHY STATUS          CREATED-TIME                 
helloworld      helloworld      webservice      scaler,hpa      running healthy Ready:1/1       2022-12-24 11:57:17 +0800 CST
ZBMac-C02CJ6QKM:~ $ 


ZBMac-C02CJ6QKM:~ $ kubectl get applications helloworld -o yaml
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"core.oam.dev/v1beta1","kind":"Application","metadata":{"annotations":{},"name":"helloworld","namespace":"default"},"spec":{"components":[{"name":"helloworld","properties":{"cpu":"0.5","exposeType":"ClusterIP","image":"oamdev/hello-world","memory":"1024Mi","ports":[{"expose":true,"port":80,"protocol":"TCP"}]},"traits":[{"properties":{"replicas":1},"type":"scaler"},{"properties":{"cpu":{"type":"Utilization","value":80},"max":10,"mem":{"type":"AverageValue","value":90},"min":1,"podCustomMetrics":[{"name":"pod_net_received_rate","value":"77"},{"name":"pod_net_transmitted_rate","value":"88"},{"name":"pod_net_received_packets_rate","value":"95"},{"name":"pod_net_transmitted_packets_rate","value":"99"}],"targetAPIVersion":"apps/v1","targetKind":"Deployment"},"type":"hpa"}],"type":"webservice"}]}}
    oam.dev/kubevela-version: v1.6.5
  creationTimestamp: "2022-12-26T03:55:31Z"
  finalizers:
  - app.oam.dev/resource-tracker-finalizer
  generation: 1
  name: helloworld
  namespace: default
  resourceVersion: "454090"
  uid: dfcf6ea0-37d1-4e93-b8df-f3a06136c8c3
spec:
  components:
  - name: helloworld
    properties:
      cpu: "0.5"
      exposeType: ClusterIP
      image: oamdev/hello-world
      memory: 1024Mi
      ports:
      - expose: true
        port: 80
        protocol: TCP
    traits:
    - properties:
        replicas: 1
      type: scaler
    - properties:
        cpu:
          type: Utilization
          value: 80
        max: 10
        mem:
          type: AverageValue
          value: 90
        min: 1
        podCustomMetrics:
        - name: pod_net_received_rate
          value: "77"
        - name: pod_net_transmitted_rate
          value: "88"
        - name: pod_net_received_packets_rate
          value: "95"
        - name: pod_net_transmitted_packets_rate
          value: "99"
        targetAPIVersion: apps/v1
        targetKind: Deployment
      type: hpa
    type: webservice
status:
  appliedResources:
  - apiVersion: apps/v1
    creator: workflow
    kind: Deployment
    name: helloworld
    namespace: default
  - apiVersion: v1
    creator: workflow
    kind: Service
    name: helloworld
    namespace: default
  - apiVersion: autoscaling/v2beta2
    creator: workflow
    kind: HorizontalPodAutoscaler
    name: helloworld
    namespace: default
  conditions:
  - lastTransitionTime: "2022-12-26T03:55:31Z"
    reason: Available
    status: "True"
    type: Parsed
  - lastTransitionTime: "2022-12-26T03:55:31Z"
    reason: Available
    status: "True"
    type: Revision
  - lastTransitionTime: "2022-12-26T03:55:31Z"
    reason: Available
    status: "True"
    type: Policy
  - lastTransitionTime: "2022-12-26T03:55:31Z"
    reason: Available
    status: "True"
    type: Render
  - lastTransitionTime: "2022-12-26T03:55:36Z"
    reason: Available
    status: "True"
    type: Workflow
  - lastTransitionTime: "2022-12-26T03:55:36Z"
    reason: ReconcileSuccess
    status: "True"
    type: Ready
  latestRevision:
    name: helloworld-v1
    revision: 1
    revisionHash: 44e6e03f3e929732
  observedGeneration: 1
  services:
  - healthy: true
    message: Ready:1/1
    name: helloworld
    namespace: default
    traits:
    - healthy: true
      type: scaler
    - healthy: true
      type: hpa
    workloadDefinition:
      apiVersion: apps/v1
      kind: Deployment
  status: running
  workflow:
    appRevision: helloworld-v1
    contextBackend:
      name: workflow-helloworld-context
      namespace: default
      uid: 673d669b-f1e7-4aa0-a9d3-b83a75980409
    endTime: "2022-12-26T03:55:36Z"
    finished: true
    mode: DAG-DAG
    startTime: "2022-12-26T03:55:31Z"
    status: succeeded
    steps:
    - firstExecuteTime: "2022-12-26T03:55:32Z"
      id: rghar81mwn
      lastExecuteTime: "2022-12-26T03:55:36Z"
      name: helloworld
      phase: succeeded
      type: apply-component
    suspend: false
    terminated: false

6.check HPA resource can be generated normally

ZBMac-C02CJ6QKM:~ $ kubectl get hpa helloworld -o yaml
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  annotations:
    app.oam.dev/last-applied-configuration: '{"apiVersion":"autoscaling/v2beta2","kind":"HorizontalPodAutoscaler","metadata":{"annotations":{"oam.dev/kubevela-version":"v1.6.5"},"labels":{"app.oam.dev/app-revision-hash":"44e6e03f3e929732","app.oam.dev/appRevision":"helloworld-v1","app.oam.dev/cluster":"","app.oam.dev/component":"helloworld","app.oam.dev/name":"helloworld","app.oam.dev/namespace":"default","app.oam.dev/resourceType":"TRAIT","app.oam.dev/revision":"helloworld-v1","oam.dev/render-hash":"74f8d484bba4a258","trait.oam.dev/resource":"hpa","trait.oam.dev/type":"hpa"},"name":"helloworld","namespace":"default"},"spec":{"maxReplicas":10,"metrics":[{"resource":{"name":"cpu","target":{"averageUtilization":80,"type":"Utilization"}},"type":"Resource"},{"resource":{"name":"memory","target":{"averageUtilization":80,"type":"AverageValue"}},"type":"Resource"},{"pods":{"metric":{"name":"pod_net_received_rate"},"target":{"averageValue":"77","type":"AverageValue"}},"type":"Pods"},{"pods":{"metric":{"name":"pod_net_transmitted_rate"},"target":{"averageValue":"88","type":"AverageValue"}},"type":"Pods"},{"pods":{"metric":{"name":"pod_net_received_packets_rate"},"target":{"averageValue":"95","type":"AverageValue"}},"type":"Pods"},{"pods":{"metric":{"name":"pod_net_transmitted_packets_rate"},"target":{"averageValue":"99","type":"AverageValue"}},"type":"Pods"}],"minReplicas":1,"scaleTargetRef":{"apiVersion":"apps/v1","kind":"Deployment","name":"helloworld"}}}'
    app.oam.dev/last-applied-time: "2022-12-26T03:55:31Z"
    autoscaling.alpha.kubernetes.io/conditions: '[{"type":"AbleToScale","status":"True","lastTransitionTime":"2022-12-26T03:56:31Z","reason":"SucceededGetScale","message":"the
      HPA controller was able to get the target''s current scale"},{"type":"ScalingActive","status":"False","lastTransitionTime":"2022-12-26T03:56:31Z","reason":"FailedGetPodsMetric","message":"the
      HPA was unable to compute the replica count: unable to get metric pod_net_received_rate:
      unable to fetch metrics from custom metrics API: no custom metrics API (custom.metrics.k8s.io)
      registered"}]'
    autoscaling.alpha.kubernetes.io/metrics: '[{"type":"Resource","resource":{"name":"memory","targetAverageUtilization":80}},{"type":"Pods","pods":{"metricName":"pod_net_received_rate","targetAverageValue":"77"}},{"type":"Pods","pods":{"metricName":"pod_net_transmitted_rate","targetAverageValue":"88"}},{"type":"Pods","pods":{"metricName":"pod_net_received_packets_rate","targetAverageValue":"95"}},{"type":"Pods","pods":{"metricName":"pod_net_transmitted_packets_rate","targetAverageValue":"99"}}]'
    oam.dev/kubevela-version: v1.6.5
  creationTimestamp: "2022-12-26T03:55:31Z"
  labels:
    app.oam.dev/app-revision-hash: 44e6e03f3e929732
    app.oam.dev/appRevision: helloworld-v1
    app.oam.dev/cluster: ""
    app.oam.dev/component: helloworld
    app.oam.dev/name: helloworld
    app.oam.dev/namespace: default
    app.oam.dev/resourceType: TRAIT
    app.oam.dev/revision: helloworld-v1
    oam.dev/render-hash: 74f8d484bba4a258
    trait.oam.dev/resource: hpa
    trait.oam.dev/type: hpa
  name: helloworld
  namespace: default
  resourceVersion: "454262"
  uid: 12d40fb2-4be7-453f-8404-f1be3f7cce08
spec:
  maxReplicas: 10
  minReplicas: 1
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: helloworld
  targetCPUUtilizationPercentage: 80
status:
  currentReplicas: 1
  desiredReplicas: 0

Special notes for your reviewer

codecov · 2022-12-24T04:09:14Z

Codecov Report

Base: 49.67% // Head: 46.56% // Decreases project coverage by -3.11% ⚠️

Coverage data is based on head (a60f114) compared to base (cadaa99).
Patch has no changes to coverable lines.

❗ Current head a60f114 differs from pull request most recent head 816a68d. Consider uploading reports for the commit 816a68d to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5225      +/-   ##
==========================================
- Coverage   49.67%   46.56%   -3.12%     
==========================================
  Files         304      249      -55     
  Lines       45663    41252    -4411     
==========================================
- Hits        22682    19208    -3474     
+ Misses      20607    19885     -722     
+ Partials     2374     2159     -215

Flag	Coverage Δ
apiserver-unittests	`36.87% <ø> (ø)`
core-unittests	`55.21% <ø> (+0.03%)`	⬆️
e2e-multicluster-test	`?`
e2e-rollout-tests	`?`
e2etests	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pkg/workflow/workflow.go	`0.00% <0.00%> (-100.00%)`	⬇️
pkg/controller/utils/actions.go	`0.00% <0.00%> (-80.00%)`	⬇️
pkg/resourcekeeper/containsresources.go	`0.00% <0.00%> (-80.00%)`	⬇️
pkg/auth/identity.go	`0.00% <0.00%> (-76.06%)`	⬇️
...v/v1alpha2/componentdefinition/mutating_handler.go	`0.00% <0.00%> (-74.29%)`	⬇️
pkg/oam/auxiliary.go	`15.21% <0.00%> (-71.74%)`	⬇️
pkg/utils/json.go	`0.00% <0.00%> (-57.15%)`	⬇️
pkg/utils/jwt.go	`0.00% <0.00%> (-55.00%)`	⬇️
pkg/resourcetracker/tree.go	`18.49% <0.00%> (-53.97%)`	⬇️
pkg/auth/kubeconfig.go	`0.00% <0.00%> (-52.85%)`	⬇️
... and 123 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

vela-templates/definitions/usage-examples/application-with-hpa.yaml

wonderflow

generally lgtm, please fix CI

StevenLeiZhang · 2022-12-26T07:51:48Z

generally lgtm, please fix CI

Done

wonderflow · 2022-12-27T08:04:24Z

Please check the comment on example doc.

jingliming · 2022-12-27T08:26:27Z

Multiple versions support of the hpa is required.
from k8s doc:
The autoscaling/v2beta2 API version of HorizontalPodAutoscaler is no longer served as of v1.26
The autoscaling/v2beta1 API version of HorizontalPodAutoscaler is no longer served as of v1.25.

https://kubernetes.io/docs/reference/using-api/deprecation-guide/

wonderflow · 2022-12-27T08:29:02Z

Multiple versions support of the hpa is required. from k8s doc: The autoscaling/v2beta2 API version of HorizontalPodAutoscaler is no longer served as of v1.26 The autoscaling/v2beta1 API version of HorizontalPodAutoscaler is no longer served as of v1.25.

https://kubernetes.io/docs/reference/using-api/deprecation-guide/

Good catch! You can refer to context.clusterVersion for this job. \cc @StevenLeiZhang

StevenLeiZhang · 2022-12-28T07:13:37Z

Multiple versions support of the hpa is required. from k8s doc: The autoscaling/v2beta2 API version of HorizontalPodAutoscaler is no longer served as of v1.26 The autoscaling/v2beta1 API version of HorizontalPodAutoscaler is no longer served as of v1.25.
https://kubernetes.io/docs/reference/using-api/deprecation-guide/

Good catch! You can refer to context.clusterVersion for this job. \cc @StevenLeiZhang

The K8s doc says

The autoscaling/v2beta2 API version of HorizontalPodAutoscaler is no longer served as of v1.26.

Migrate manifests and API clients to use the autoscaling/v2 API version, available since v1.23.
All existing persisted objects are accessible via the new API

so I add

		if context.clusterVersion.minor < 23 {
			apiVersion: "autoscaling/v2beta2"
		}
		if context.clusterVersion.minor >= 23 {
			apiVersion: "autoscaling/v2"
		}

is it Ok? \cc @jingliming

jingliming · 2022-12-29T10:46:53Z

I remembered the autoscaling/v2beta1 supports k8s 1.7+ version，but not sure which minimum version the autoscaling/v2beta2 supports，It might be better to add minimum version support

references/docgen/def-doc/trait/hpa.eg.md

wonderflow

LGTM after the example doc fixed.

…M/PodCustomMetrcs Signed-off-by: StevenLeiZhang <zhangleiic@163.com>

…M/PodCustomMetrcs (kubevela#5225) Signed-off-by: StevenLeiZhang <zhangleiic@163.com> Signed-off-by: StevenLeiZhang <zhangleiic@163.com>

StevenLeiZhang requested review from Somefive, barnettZQG, wonderflow and FogDong as code owners December 24, 2022 04:00

StevenLeiZhang force-pushed the tt-hpa branch 3 times, most recently from a60f114 to 9cbe01e Compare December 24, 2022 09:19

wonderflow reviewed Dec 26, 2022

View reviewed changes

vela-templates/definitions/usage-examples/application-with-hpa.yaml Outdated Show resolved Hide resolved

wonderflow reviewed Dec 26, 2022

View reviewed changes

StevenLeiZhang requested a review from wonderflow December 26, 2022 03:12

StevenLeiZhang force-pushed the tt-hpa branch from 9cbe01e to 29c5871 Compare December 26, 2022 04:07

StevenLeiZhang requested review from leejanee and jefree-cat as code owners December 26, 2022 04:07

StevenLeiZhang force-pushed the tt-hpa branch 2 times, most recently from 59a4320 to 11ccd1f Compare December 26, 2022 07:23

Somefive approved these changes Dec 27, 2022

View reviewed changes

StevenLeiZhang force-pushed the tt-hpa branch from 11ccd1f to 2a1cdd3 Compare December 28, 2022 07:10

wonderflow reviewed Dec 30, 2022

View reviewed changes

references/docgen/def-doc/trait/hpa.eg.md Show resolved Hide resolved

wonderflow reviewed Dec 30, 2022

View reviewed changes

Feat: need one new Trait to support HorizontalPodAutoscaler of CPU/ME…

816a68d

…M/PodCustomMetrcs Signed-off-by: StevenLeiZhang <zhangleiic@163.com>

StevenLeiZhang force-pushed the tt-hpa branch from 2a1cdd3 to 816a68d Compare December 30, 2022 12:33

wonderflow approved these changes Jan 3, 2023

View reviewed changes

wonderflow merged commit 8a52395 into kubevela:master Jan 3, 2023

jefree-cat mentioned this pull request Mar 9, 2023

Nominate StevenLeiZhang as a maintainer of KubeVela Org kubevela/community#128

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat: need one new Trait to support HorizontalPodAutoscaler of CPU/ME… #5225

Feat: need one new Trait to support HorizontalPodAutoscaler of CPU/ME… #5225

StevenLeiZhang commented Dec 24, 2022 •

edited

codecov bot commented Dec 24, 2022 •

edited

wonderflow left a comment

StevenLeiZhang commented Dec 26, 2022

wonderflow commented Dec 27, 2022

jingliming commented Dec 27, 2022

wonderflow commented Dec 27, 2022

StevenLeiZhang commented Dec 28, 2022

jingliming commented Dec 29, 2022 •

edited

wonderflow left a comment

Feat: need one new Trait to support HorizontalPodAutoscaler of CPU/ME… #5225

Feat: need one new Trait to support HorizontalPodAutoscaler of CPU/ME… #5225

Conversation

StevenLeiZhang commented Dec 24, 2022 • edited

Description of your changes

How has this code been tested

Special notes for your reviewer

codecov bot commented Dec 24, 2022 • edited

Codecov Report

wonderflow left a comment

Choose a reason for hiding this comment

StevenLeiZhang commented Dec 26, 2022

wonderflow commented Dec 27, 2022

jingliming commented Dec 27, 2022

wonderflow commented Dec 27, 2022

StevenLeiZhang commented Dec 28, 2022

jingliming commented Dec 29, 2022 • edited

wonderflow left a comment

Choose a reason for hiding this comment

StevenLeiZhang commented Dec 24, 2022 •

edited

codecov bot commented Dec 24, 2022 •

edited

jingliming commented Dec 29, 2022 •

edited