Skip to content
This repository has been archived by the owner on Jul 23, 2020. It is now read-only.

Maven artifactIds starting with a number can't rollout from Jenkins #3784

Closed
jiekang opened this issue Jun 13, 2018 · 21 comments
Closed

Maven artifactIds starting with a number can't rollout from Jenkins #3784

jiekang opened this issue Jun 13, 2018 · 21 comments

Comments

@jiekang
Copy link
Collaborator

jiekang commented Jun 13, 2018

EDIT:

Scenario 2 steps to reproduce:

  1. Import application from GitHub repository with maven project name that starts with a number.
  2. Observe build failure in Jenkins:
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://kubernetes.default/api/v1/namespaces/jkang-stage/services. Message: Service "1234-4321" is invalid: metadata.name: Invalid value: "1234-4321": a DNS-1035 label must consist of lower case alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character (e.g. 'my-name',  or 'abc-123', regex used for validation is '[a-z]([-a-z0-9]*[a-z0-9])?'). Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=metadata.name, message=Invalid value: "1234-4321": a DNS-1035 label must consist of lower case alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character (e.g. 'my-name',  or 'abc-123', regex used for validation is '[a-z]([-a-z0-9]*[a-z0-9])?'), reason=FieldValueInvalid, additionalProperties={})], group=null, kind=Service, name=1234-4321, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Service "1234-4321" is invalid: metadata.name: Invalid value: "1234-4321": a DNS-1035 label must consist of lower case alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character (e.g. 'my-name',  or 'abc-123', regex used for validation is '[a-z]([-a-z0-9]*[a-z0-9])?'), metadata=ListMeta(resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:470)

More specifically:

Invalid value: "1234-4321": a DNS-1035 label must consist of lower case alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character
@hrishin
Copy link

hrishin commented Jun 15, 2018

Would it be better to add checks from launcher backend as well?

@gastaldi @animuk

@gastaldi
Copy link
Member

gastaldi commented Jun 15, 2018

@hrishin agreed. We would be really happy if you would send us a PR ;)

@hrishin
Copy link

hrishin commented Jun 15, 2018

@gastaldi sent a PR 👍 . Could you please review it?

@hrishin
Copy link

hrishin commented Jun 18, 2018

@jiekang still it's not working. (Is it in production?)

1123213-1232": a DNS-1035 label must consist of lower case alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character (e.g. 'my-name', or 'abc-123', regex used for validation is '[a-z]([-a-z0-9]*[a-z0-9])?')

Can we use this [a-z]([-a-z0-9]*[a-z0-9])?' regex as it is for validation or normalize project name according to regex?
it forces to use small case only.

Though it would be very restrictive for application/project names.

@jiekang
Copy link
Collaborator Author

jiekang commented Jun 18, 2018

I think the change in fabric8-ui is not in production yet; after it is included, a user should not be able to create a project starting with a number.

A better change throughout the whole project is to keep names as unrestricted strings that users can set to whatever, and use an internal 'name' or 'uuid' for the key that is valid for the different APIs we use: Jenkins, Kubernetes, Openshift, etc.

@jiekang
Copy link
Collaborator Author

jiekang commented Jun 18, 2018

Though the UI does not allow application names with a number after fabric8-ui/fabric8-ui#3025, this is still an issue as it seems to be using the GitHub repository name.

  1. Create GitHub repository with name starting with number, like 1234-4321.
  2. Import codebase with valid application name abcd and use GitHub repository above.
  3. Observe failure to build with same error posted in the issue details.

[...]Message: Service "1234-4321" is invalid: metadata.name: Invalid value: "1234-4321": a DNS-1035 label must consist of lower case alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character [...]

@hrishin hrishin assigned jiekang and unassigned hrishin Jun 21, 2018
@jiekang
Copy link
Collaborator Author

jiekang commented Jun 21, 2018

@hrishin why is this assigned to me? The issue is not in fabric8-ui, it's in the build system.

@jiekang jiekang changed the title Applications starting with a number can't rollout from Jenkins Applications or GitHub repositories starting with a number can't rollout from Jenkins Jun 21, 2018
@hrishin hrishin assigned hrishin and unassigned jiekang Jun 22, 2018
@hrishin
Copy link

hrishin commented Jun 22, 2018

@jiekang no worries. This issue could be addressed at various levels. It's launcher first that has to validate project name to compliment DNS name constraints. At build side, there is little/no scope to address this it.

@gastaldi @animuk

@joshuawilson
Copy link
Member

@hrishin we do not assign people.

Assignment == commitment to do the work.

So only the person doing the work should assign themselves when they take it on. The only except is the Lead may assign them during planning.

@jiekang jiekang changed the title Applications or GitHub repositories starting with a number can't rollout from Jenkins Application Names or Maven project names starting with a number can't rollout from Jenkins Jun 22, 2018
@jiekang jiekang changed the title Maven project names starting with a number can't rollout from Jenkins Maven artifactIds starting with a number can't rollout from Jenkins Jun 22, 2018
@jiekang
Copy link
Collaborator Author

jiekang commented Jun 22, 2018

So after a little more digging with a vert.x http booster GitHub repository that was renamed to start with a number, I have found that it's actually the maven artifactId that triggers this issue. A pipeline with just the GitHub name change passed. After changing the artifactId to match the name, the pipeline failed. For an example see: https://github.com/jiekang/1234-4321/commits/master (This repository may not exist in a week from this comment)

With validation in place on f8-ui side, it's no longer possible for users to use the wizard to have a booster repository created that will trigger this, but for custom projects, it may be an issue. Though I don't recall many projects ever having an artifactId starting with a number, I can't find any clear documentation on whether or not it's allowed. From [1], it does seem valid, albeit out of the ordinary.

I think it would be best for the Jenkins build system calls to stop using raw strings coming from elsewhere. Maybe it would be okay to apply a lowercase function, and append 'kube-' to all inputs?

[1] https://maven.apache.org/guides/mini/guide-naming-conventions.html

@hrishin
Copy link

hrishin commented Jun 22, 2018

@joshuawilson noted, will keep that in mind 👍

@jiekang thanks a lot for the further investigation on this. 👍

If a user commits artifacts id with an invalid name it would boil down to build system. yeah, let's create the service name by appending service- with all lowercase. Let's see if it works.

@hrishin
Copy link

hrishin commented Jun 28, 2018

Added fix for this issue jenkinsci/kubernetes-pipeline-plugin#68

hrishin added a commit to hrishin/openshift-jenkins-s2i-config that referenced this issue Jul 12, 2018
…kins

For most of the applications, service name is derived from application name. If service name is not
adhering to valid DNS characters then applyKubernetes step fails to create k8s resources. To fix this
issue, this patch will update(sanitize) the invalid service name by prefixing `svc-`, transforms all
characters to small case and updates the service reference in route. This auto-patching can be enabled
or disabled by setting ENV var "K8S_PIPELINE_SERVICE_PATCH" to "enabled" or "K8S_PIPELINE_SERVICE_PATCH"
to "disabled" (by default its disabled)".
Patch has been fixed in kubernetes-pipeline-plugin -> devops project.

Changes:
- Plugin file built from jenkinsci/kubernetes-pipeline-plugin@e89bc40
- Renamed plugin file from hpi to jpi as jpi has higher precedence
- Update version.txt

Fixes openshiftio/openshift.io#3784
hrishin added a commit to hrishin/fabric8-tenant-jenkins that referenced this issue Jul 12, 2018
For most of the applications, service name is derived from application name. If service name is not
adhering to valid DNS characters then applyKubernetes step fails to create k8s resources. To fix this
issue, this patch will update(sanitize) the invalid service name by prefixing , transforms all
characters to small case and updates the service reference in route. This auto-patching can be enabled
or disabled by setting ENV var K8S_PIPELINE_SERVICE_PATCH to enabled or K8S_PIPELINE_SERVICE_PATCH
to disabled (by default its disabled).

Changes:
- Added `K8S_PIPELINE_SERVICE_PATCH` ENV VAR in kubernetes and openshift deployments

Fixes openshiftio/openshift.io#3784
sthaha pushed a commit to fabric8io/openshift-jenkins-s2i-config that referenced this issue Jul 12, 2018
For most of the applications, service name is derived from application name. If service name is not
adhering to valid DNS characters then applyKubernetes step fails to create k8s resources. To fix this
issue, this patch updates (sanitizes) the invalid service name by prefixing `svc-`, transforms all
characters to small case and updates the service reference in route. This auto-patching can be toggled
by setting ENV var `K8S_PIPELINE_SERVICE_PATCH` to `enabled` or to  `disabled` (default).
Patch has been added in kubernetes-pipeline-plugin -> devops project.

Changes:
- Plugin has been built from jenkinsci/kubernetes-pipeline-plugin@e89bc40
- Renamed plugin file from hpi to jpi as jpi takes higher precedence
- Update version.txt

Fixes openshiftio/openshift.io#3784
@hrishin
Copy link

hrishin commented Jul 12, 2018

got closed due to PR merge

@hrishin hrishin reopened this Jul 12, 2018
sthaha pushed a commit to fabric8-services/fabric8-tenant-jenkins that referenced this issue Jul 12, 2018
For most of the applications, service name is derived from application name. If service name is not
adhering to valid DNS characters then applyKubernetes step fails to create k8s resources. To fix this
issue, this patch updates (sanitizes) the invalid service name by prefixing `svc-`, transforms all
characters to small case and updates the service reference in route. This auto-patching can be toggled
by setting ENV var `K8S_PIPELINE_SERVICE_PATCH` to `enabled` or to  `disabled` (default).

Changes:
- Added `K8S_PIPELINE_SERVICE_PATCH` ENV VAR in kubernetes and openshift deployments

Fixes openshiftio/openshift.io#3784
@jiekang
Copy link
Collaborator Author

jiekang commented Jul 26, 2018

I tried to rollout to stage:

Application Name: awtftwtaw
Repository Name: 1234-4321
Maven Artifact ID: 1234-4321

and still see failure log:

ERROR: Error during kubernetes apply: java.lang.RuntimeException: Failed to create Service from pipeline.json. io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://kubernetes.default/api/v1/namespaces/jkang-stage/services. Message: Service "1234-4321" is invalid: metadata.name: Invalid value: "1234-4321": a DNS-1035 label must consist of lower case alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character (e.g. 'my-name',  or 'abc-123', regex used for validation is '[a-z]([-a-z0-9]*[a-z0-9])?'). Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=metadata.name, message=Invalid value: "1234-4321": a DNS-1035 label must consist of lower case alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character (e.g. 'my-name',  or 'abc-123', regex used for validation is '[a-z]([-a-z0-9]*[a-z0-9])?'), reason=FieldValueInvalid, additionalProperties={})], group=null, kind=Service, name=1234-4321, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Service "1234-4321" is invalid: metadata.name: Invalid value: "1234-4321": a DNS-1035 label must consist of lower case alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character (e.g. 'my-name',  or 'abc-123', regex used for validation is '[a-z]([-a-z0-9]*[a-z0-9])?'), metadata=ListMeta(resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).. Service(apiVersion=v1, kind=Service, metadata=ObjectMeta(annotations={fabric8.io/git-commit=ef21ddaefb9bfc0568555d1f65323520aef9ad67, fabric8.io/build-id=1, fabric8.io/build-url=http://jenkins-jkang-jenkins.8a09.starter-us-east-2.openshiftapps.com/job/booster-parent/1, prometheus.io/port=9779, fabric8.io/iconUrl=img/icons/vertx.svg, fabric8.io/git-branch=jiekang/1234-4321/master-1.0.1, prometheus.io/scrape=true, fabric8.io/git-url=gogsadmin/booster-parent/commit/ef21ddaefb9bfc0568555d1f65323520aef9ad67}, clusterName=null, creationTimestamp=null, deletionGracePeriodSeconds=null, deletionTimestamp=null, finalizers=[], generateName=null, generation=null, initializers=null, labels={expose=true, space=abcd_efgh, app=1234-4321, provider=fabric8, version=1.0.1, group=io.openshift.booster}, name=1234-4321, namespace=null, ownerReferences=[], resourceVersion=null, selfLink=null, uid=null, additionalProperties={}), spec=ServiceSpec(clusterIP=null, externalIPs=[], externalName=null, externalTrafficPolicy=null, healthCheckNodePort=null, loadBalancerIP=null, loadBalancerSourceRanges=[], ports=[ServicePort(name=http, nodePort=null, port=8080, protocol=TCP, targetPort=IntOrString(IntVal=8080, Kind=null, StrVal=null, additionalProperties={}), additionalProperties={})], selector={app=1234-4321, provider=fabric8, group=io.openshift.booster}, sessionAffinity=null, type=null, additionalProperties={}), status=null, additionalProperties={})
	at io.fabric8.kubernetes.api.Controller.onApplyError(Controller.java:1459)
	at io.fabric8.kubernetes.api.Controller.doCreateService(Controller.java:1103)
	at io.fabric8.kubernetes.api.Controller.applyService(Controller.java:1030)
	at io.fabric8.kubernetes.pipeline.devops.ApplyStepExecution.run(ApplyStepExecution.java:218)
	at io.fabric8.kubernetes.pipeline.devops.ApplyStepExecution.run(ApplyStepExecution.java:104)

@jiekang jiekang reopened this Jul 26, 2018
@jiekang
Copy link
Collaborator Author

jiekang commented Jul 26, 2018

@hrishin @sthaha Any ideas? Is the update not in production or maybe something is still missing?

@hrishin
Copy link

hrishin commented Jul 26, 2018

@jiekang the Its not updated in prod yet.

@jiekang
Copy link
Collaborator Author

jiekang commented Jul 26, 2018

What's the plan to get it into production? Is there anything blocking it? If it's all good feel free to close this issue again.

@hrishin
Copy link

hrishin commented Jul 27, 2018

Is there anything blocking it.

yesterday we were trying to roll out into prod but we are facing issues with builds, hence it is blocked. Will post the update here.

@hrishin
Copy link

hrishin commented Jul 27, 2018

Just for info: Test build logs https://pastebin.com/3TRm3eUP for the repo https://github.com/hrishin/1222-123236435141.
This is how patched service URL looks
svc-1222-123236435141: "https://svc-1222-123236435141-hriships-stage.b542.starter-us-east-2a.openshiftapps.com"

This update is blocked due to #4077

@jiekang
Copy link
Collaborator Author

jiekang commented Jul 27, 2018

Okay, thanks for the updates!

@piyush-garg
Copy link
Collaborator

@jiekang @hrishin This has been promoted to prod in tenant Jenkins version 4.0.95. Thanks, closing this.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.