[Feature]: App pusher can specify a particular buildpack from the available list

[kieron-dev]

Blockers/Dependencies

This will address bug #1784

Background

As a app developer
I want to specify building using a particular buildpack from the ones available
So that I can address problems where the wrong buildpack is selected for the build.

The list of available buildpack is retrieved using the cf buildpacks command:

$ cf buildpacks
Getting buildpacks as cf-admin...

position   name                         stack                         enabled   locked   filename
1          paketo-buildpacks/java       io.buildpacks.stacks.bionic   true      false    paketo-buildpacks/java@9.4.0
2          paketo-buildpacks/go         io.buildpacks.stacks.bionic   true      false    paketo-buildpacks/go@4.3.3
3          paketo-buildpacks/nodejs     io.buildpacks.stacks.bionic   true      false    paketo-buildpacks/nodejs@1.6.0
4          paketo-buildpacks/ruby       io.buildpacks.stacks.bionic   true      false    paketo-buildpacks/ruby@0.34.0
5          paketo-buildpacks/procfile   io.buildpacks.stacks.bionic   true      false    paketo-buildpacks/procfile@5.6.1

There are instances where the presence of a certain file may confuse the automatic buildpack selection. The app developer can fix this themselves by specifying the -b or --buildpack option for cf push.

It is also possible to specify buildpacks in an app manifest using the (deprecated) buildpack field, or the buildpacks list field. This should be supported in the same way as using the cf push --buildpack option.

Note: this feature will allow selecting an existing buildpack only; custom buildpacks specified by a git URL are out of scope.

Acceptance Criteria

GIVEN a buildpack exists with the name 'repo/my-buildpack'
WHEN I cf push my app using the option -b repo/my-buildpack or --buildpack repo/my-buildpack
THEN I see the app was built using the buildpack specified

GIVEN a buildpack exists with the name 'repo/my-buildpack'
WHEN I cf push my app using a manifest with buildpack set to 'repo/my-buildpack' or buildpacks set to a list of ['repo/my-buildpack']
THEN I see the app was built using the buildpack specified

GIVEN buildpacks exists with names 'repo/my-buildpack', 'repo/another-buildpack'
AND 'repo/another-buildpack' will not build my app by 'repo/my-buildpack' will
WHEN I cf push my app using a manifest with buildpacks set to a list of ['repo/another-buildpack', 'repo/my-buildpack']
THEN I see the app was built trying the buildpacks specified

Dev Notes

When the kpack-image-builder reconciles a BuildWorkload resource with non-empty buildpacks list, it should:

1. Check the specified buildpacks exist in the default ClusterStore. If not, set a failed status on the BuildWorkload with an appropriate error message
2. Compute a hash of the buildpacks list to use as a Builder name. Ordering is important.
3. Compute the OCI image tag for the Builder. This should reuse the repository prefix used for droplets, appending 'builders-' and the builder name. Ensure the OCI repository is created if necessary using the same mechanism as for droplets.
4. If a Builder already exists in the CF Space's namespace with the computed name, use it. Otherwise create a new kpack Builder with the computed name, reusing the serviceAccountRef, stack, and store from the default ClusterBuilder. The order field should contain the list of buildpack IDs in the format

   order:
     - group:
       - id: buildpack-1
     - group:
       - id: buildpack-2
   

5. Set the BuildWorkload as an owner on the newly created or already existing Builder, so that it will be garbage collected when all BuildWorkload references have been deleted.
6. Use the Builder name on the kpack Image created by the reconciler. The initialisation phase of the reconciler will exit at this point, and the build process should process normally after that. It will wait for the Builder to be ready if necessary.

[kieron-dev]

The normal use case for this feature is:

1. User does a cf push using the default ClusterBuilder buildpack order, but the wrong buildpack is detected.
2. User specifies the correct buildpack using -b option to cf push.

Step 1 results in a failed kpack.Build on the kpack.Image associated with the CFApp. Step 2 modifies the kpack.Image to point to a new Builder with just the desired buildpack in it, but kpack Image reconciler logic says this should not trigger a new Build because the lastBuild status is failed: https://github.com/pivotal/kpack/blob/9081a1861d6551e86110653f80259c9a17c2b28f/pkg/reconciler/image/build_required.go#L108.

I've raised an issue with kpack here: buildpacks-community/kpack#1198. I think we need to block this feature on that.