Fail to create task after delete credentials

[lizzzcai]

Hi, I am trying to create a Build CR with output credentials (do-secret) to a private registry. The buildRun is working fine.

After that, I removed the credentials (do-secret) from my build CR, the buildRun still working. (somewhere my credential was cached).

If I deleted my credentials (do-secret), the buildRun is failed (I didn't mention the credential in the Build CR already). Below is the error:

  conditions:
  - lastTransitionTime: "2021-12-23T14:43:53Z"
    message: 'failed to create task run pod "buildpack-go-buildrun-qbgq7-tr6sp": translating
      TaskSpec to Pod: secrets "do-secret" not found. Maybe invalid TaskSpec'
    reason: CouldntGetTask
    status: "False"
    type: Succeeded
  latestTaskRunRef: buildpack-go-buildrun-qbgq7-tr6sp

All my buildRun failed after I delete the secret.

[adambkaplan]

/kind question

[openshift-ci]

@adambkaplan: The label(s) kind/question cannot be applied, because the repository doesn't have them.

In response to this:

/kind question

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[adambkaplan]

/assign @SaschaSchwarze0

[SaschaSchwarze0]

Hi @lizzzcai, thank you for trying Shipwright. You hit one of our known weak points. :-)

What happens is that when you create a BuildRun, the referenced container registry secret (mainly .spec.output.image.credentials.name) gets still passed to the build strategy code through Tekton's creds-init machinery. This requires us to add the secret to a ServiceAccount that we set on the Tekton TaskRun. But, we never remove a Secret from a ServiceAccount. Therefore, deleting a secret is a problem. What you can do is to remove the secret manually:

$ kubectl -n <MY_NAMESPACE> edit serviceaccount <SERVICEACCOUNT_NAME>

The service account name should be default assuming you never specified something related to service accounts in your BuildRun. From the list of secrets, you can remove those that you deleted.

To prevent this problem from happening, you can specify BuildRuns in the following way:

apiVersion: shipwright.io/v1alpha1
kind: BuildRun
metadata:
  name: <BUILDRUN_NAME>
spec:
  buildRef:
    name: <BUILD_NAME>
  serviceAccount:
    generate: true

Then, Shipwright will create a dedicated ServiceAccount for your BuildRun and delete it when it completes.

In case you use the shp CLI, then --sa-generate is the command line flag that you need to specify when submitting the BuildRun.

Mid-term this whole problem will be gone as we will move towards getting rid of the creds-init usage. This is captured in As a shipwright developer, I want to stop relying on service accounts for mounting my secrets at the pod level #679.

[lizzzcai]

Thanks @SaschaSchwarze0 , I have tested it and it is working. I will close the issue.