[Issue 9360][pulsar-functions] kubernetes runtime functions create rfc1123 compliant labels #9556
Conversation
|
/pulsarbot run-failure-checks |
1 similar comment
|
/pulsarbot run-failure-checks |
sijie
added
area/function
type/enhancement
There was a problem hiding this comment.
It would be better if the PR describes the specific rule of updating label values. I add it here for future reference:
- truncate to 63 chars
- replace any non alphanumeric character ([a-z0-9A-Z]), dashes (-), underscores(_), dot(.) with "-"
- replace beginning & end non alphanumeric character with "0"
sijie
added
the
doc-required
|
@nlu90 Great suggestion! I have updated the rules to the PR description. |
…c1123 compliant labels (#9556) Fixes #9360 ### Motivation Currently, it's valid to (via pulsar-admin and the admin rest api) specify names that include the colon ':' character. For example, I can create a namespace via pulsar-admin and the rest api as something like 'my:example:namespace'. We've found this useful for purposes of adding structure to object names that is then easier to programmatically parse. We've been able to use this format with no problems with the ProcessRuntime for deploying functions. The KubernetesRuntime, however, is currently not compatible with pulsar object names that are not compliant with RFC1123, because KubernetesRuntime attempts to use these object names verbatim in the labels it attaches to the resource specifications. Trying to use the KubernetesRuntime with the name format described above prevents the functions from deploying successfully. ### Modifications Changed the KubernetesRuntime to translate the names of the pulsar objects to forms that are RFC1123-compliant for k8s resource labels. The rules for translating the Pulsar object names are: - truncate to 63 chars - replace any non alphanumeric character ([a-z0-9A-Z]), dashes (-), underscores(_), dot(.) with "-" - replace beginning & end non-alphanumeric character with "0" This change added tests and can be verified as follows: - Added an extra test to KubernetesRuntimeTest.java to make sure labels are created that are RFC1123-compliant AND also make sure the labels are no bigger than the k8s limit of 63 chars (cherry picked from commit 5763c0e)
|
Hi @Anonymitaet, That looks good. Your suggestions for the updates are correct. I'm also wondering if you should mention or link to this document as well: https://pulsar.apache.org/docs/en/next/functions-runtime/ It contains other useful information for making sure you setup your pulsar resources correctly to work within Kubernetes without issue. You could point out the customizable options in CustomRuntimeOptions for 'JobName' and 'JobNamespace'. Having those available can also help a user not run into k8s issues with pod names being too long by default (pf---<function_name>) if you have defined long tenant, namespace, and function names in Pulsar when creating functions on the Kubernetes runtime. |
|
Done in #9651 |
Anonymitaet
removed
the
doc-required
Fixes #9360
Motivation
Currently, it's valid to (via pulsar-admin and the admin rest api) specify names that include the colon ':' character. For example, I can create a namespace via pulsar-admin and the rest api as something like 'my:example:namespace'. We've found this useful for purposes of adding structure to object names that is then easier to programmatically parse.
We've been able to use this format with no problems with the ProcessRuntime for deploying functions.
The KubernetesRuntime, however, is currently not compatible with pulsar object names that are not compliant with RFC1123, because KubernetesRuntime attempts to use these object names verbatim in the labels it attaches to the resource specifications.
Trying to use the KubernetesRuntime with the name format described above prevents the functions from deploying successfully.
Modifications
Changed the KubernetesRuntime to translate the names of the pulsar objects to forms that are RFC1123-compliant for k8s resource labels.
The rules for translating the Pulsar object names are:
This change added tests and can be verified as follows:
Does this pull request potentially affect one of the following parts:
Documentation