Using VPA with custom controllers

[svgreatest]

In my project, I have a Kubernetes custom resource with a custom controller. I was trying to use VPA with the custom controller. I'm basically following this: 8ff3b4f to add my custom resource. I need a separate clientset for my custom resource. I couldn't find any good way to hook that code into existing VPA code. Do you have any recommendation on how to go about adding a custom resource to VPA?

[bskiba]

In addition to supporting most common controllers, VPA can target custom resources without changes to code. However, your custom resource needs to support scale subresource so that VPA can find out which pods it controlls:
https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#scale-subresource

Let me know if this works for you.

[svgreatest]

I'm trying this in GKE:

bash-4.2$ ./vpa-up.sh
customresourcedefinition.apiextensions.k8s.io/verticalpodautoscalers.autoscaling.k8s.io created
customresourcedefinition.apiextensions.k8s.io/verticalpodautoscalercheckpoints.autoscaling.k8s.io created
clusterrole.rbac.authorization.k8s.io/system:metrics-reader created
clusterrole.rbac.authorization.k8s.io/system:vpa-actor created
clusterrole.rbac.authorization.k8s.io/system:vpa-checkpoint-actor created
clusterrole.rbac.authorization.k8s.io/system:evictioner created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-reader created
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-actor created
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-checkpoint-actor created
clusterrole.rbac.authorization.k8s.io/system:vpa-target-reader created
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-target-reader-binding created
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-evictionter-binding created
serviceaccount/vpa-admission-controller created
clusterrole.rbac.authorization.k8s.io/system:vpa-admission-controller created
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-admission-controller created
clusterrole.rbac.authorization.k8s.io/system:vpa-status-reader created
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-status-reader-binding created
serviceaccount/vpa-updater created
deployment.apps/vpa-updater created
serviceaccount/vpa-recommender created
deployment.apps/vpa-recommender created
Generating certs for the VPA Admission Controller in /tmp/vpa-certs.
Generating RSA private key, 2048 bit long modulus
......................................+++
...........................................................+++
e is 65537 (0x10001)
Generating RSA private key, 2048 bit long modulus
......................+++
........................+++
e is 65537 (0x10001)
Signature ok

bash-4.2$ kubectl describe vpa
Name: v1-vpa1
Namespace:
Labels:
Annotations:
API Version: autoscaling.k8s.io/v1beta2
Kind: VerticalPodAutoscaler

Message: Cannot read targetRef. Reason: Unhandled targetRef <api_group>/ / <custom resource> / <name>, last error resource.api.group "<name>" is forbidden: User "system:serviceaccount:kube-system:vpa-recommender" cannot get resource "<resource>/scale" in API group "<api group>" in the namespace "<custom namespace>"

In my CRD, I have added:
subresource:
scale:
...

kubectl get --raw /apis// | jq

{
"name": "/scale",
"singularName": "",
"namespaced": true,
"group": "autoscaling",
"version": "v1",
"kind": "Scale",
"verbs": [
"get",
"patch",
"update"
]
}

Any thoughts as to why VPA is complaining about the resource not being there?

[bskiba]

Might be permission issues. Can you try modifying vpa-rbac.yaml to enable VPA recommender to get and watch your custom resource?

[bskiba]

There is a PR in review that might fix this.

[bskiba]

#3139 should fix this. Please reopen if you still see issues

[bskiba]

/close

[k8s-ci-robot]

@bskiba: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[lokeshkumar380]

facing this error
Status:
Conditions:
Last Transition Time: 2021-02-25T05:04:24Z
Message: Cannot read targetRef. Reason: Unhandled targetRef argoproj.io/v1alpha1 / Workflow / test, last error the server could not find the requested resource (get workflows test)
Status: True
Type: ConfigUnsupported
Last Transition Time: 2021-02-25T05:04:24Z
Message: No pods match this VPA object
Reason: NoPodsMatched
Status: True
Type: NoPodsMatched
Last Transition Time: 2021-02-25T05:04:24Z
Message: No pods match this VPA object
Reason: NoPodsMatched
Status: False
Type: RecommendationProvided
Recommendation:

[mohang6770]

same error as above
Status:
Conditions:
Last Transition Time: 2022-11-06T22:04:41Z
Message: Cannot read targetRef. Reason: Deployment kube-system/my-app does not exist
Status: True
Type: ConfigUnsupported
Last Transition Time: 2022-11-06T22:04:41Z
Message: No pods match this VPA object
Reason: NoPodsMatched
Status: True
Type: NoPodsMatched
Last Transition Time: 2022-11-06T22:04:41Z
Message: No pods match this VPA object
Reason: NoPodsMatched
Status: False
Type: RecommendationProvided
Recommendation:
Events: