Introduce outerloop 'dockerfile' build & deploy

[sbose78]

(EDITED - check edit history for prior revisions and discussion of the opening post)

For devfile 2.2.0. Check the proposal at https://github.com/devfile/api/blob/main/docs/proposals/devfile/outer-loop-build-and-deploy.md

A stack may require expressing the outer-loop build guidance ( source code to image ) in the form of a Dockerfile build, s2i build, etc.

    variables:
      myimage: myimagename
    components:
      - name: mydockerfileimage
        image:
          imageName: {{myimage}}
          dockerfile:
            buildContext: ${PROJECTS_ROOT}/build
            uri: Dockerfile
            args: [ "arg1", "arg2", "arg3" ]
            rootRequired: false

The other dockerfile src are available in the proposal doc.

Once the runtime image has been built, we can deploy it via the deploy group command:

components:
  - name: outerloop-deploy
    kubernetes:
      uri: deployment-manifest.yaml
commands:
  - id: deployk8s
    apply:
      component: outerloop-deploy
      group:
        kind: deploy
        isDefault: true

The build tool mechanism like kaniko or buildah will be up to the tools and hence will not be a part of the spec.

[EnriqueL8]

Would there be an idea here to add a build image to be used to consume that Dockerfile? I am thinking of a Buildah image or a Kaniko image that would both be used locally and in a pipeline? Would this be something the stack creator should add, or the tool consuming the devfile decide?

[l0rd]

In Che, Devfile components are created before the project gets cloned: we do not have access to ${PROJECTS_ROOT}/build/Dockerfile yet when we need to create the Dockerfile component.

If we want to support this use case we need to agree on the workspace creation flow: cloning of the project should happen before everything else.

On the other hand I think we should consider publishing Dockerfiles and ohter components definitions in a devfile registry so that we may reference as:

components:
  - id: build-node-app      
    dockerfile:
        id: mycompany/my-node-stack-dockerfile/v2.2.2
        args: []

[sbose78]

In Che, Devfile components are created before the project gets cloned: we do not have access to ${PROJECTS_ROOT}/build/Dockerfile yet when we need to create the Dockerfile component.

Should we annotate this component to ensure that building this image is not a pre-req to the workspace starting up in Che?

On the other hand I think we should consider publishing Dockerfiles and ohter components definitions in a devfile registry so that we may reference as:

Sounds good.

mycompany/my-node-stack-dockerfile/v2.2.2

Could you share a bit more on how you wish the files to be laid out in the registry?

[sbose78]

• Mario: Probably a tool (odo/che) specific configuration

• Gorkem:

  • Build mechanism is not a problem for the developer.
  • If something like Buildpacks, should be a new component type.
  • kaniko/buildah needn't be specified.
  • Events: Reference the built image in another place in the devfile?

[ inheritance ]

• Elson: remove component and add new one in inheritance
• David: override component with same id, but with a different build guidance.

[elsony]

For using the id and path in a dockerfile component, I am trying to understand what this item is purposing. Are we suggesting to add id as another way to specify where the dockerfile is or are we proposing id as the only way to specify and replacing path? I can see path is useful when someone is just bringing their own devfile as part of the app (not coming from a registry) so they can put their dockerfile as part of the app as well.

[maysunfaisal]

When working on the Console Dev Preview for Build Guidance, we found some information that the devfile spec probably may need to address. There is currently an open PR on devfile/api for Build Guidance but we can base our spec on that basis - https://github.com/devfile/api/pull/127/files#diff-ce0147ba3842f44db1642f6506de48a1744cce30f46475c5df580e34e36f0a59R38

1. The Build Guidance dockerfile spec takes mostly OpenShift and ImageStreamTag into consideration. OpenShift BuildConfig allows a user to also push the image from BuildConfig to a registry other than the integrated OpenShift registry(like a private registry or Dockerhub for example) using DockerImage, and in such cases there needs to be a way to specify the push secrets - https://docs.openshift.com/container-platform/4.6/builds/managing-build-output.html and https://docs.openshift.com/container-platform/3.11/dev_guide/builds/build_inputs.html#using-docker-credentials-for-private-registries
2. Currently, the devfile Build Guidance spec has no way of providing what the output image from the BuildConfig will be called or a way to link an endpoint for the created image - The Console dev preview achieves this with a workaround by filtering components with an attribute filter console-import - https://github.com/redhat-developer/devfile-sample/blob/master/devfile.yaml#L11. One thing to note is, devfile validation and expectation does not allow endpoints across components to have the same port for endpoints. Are we expecting different endpoint ports for an application that can be run via Build Guidance and regular Devfile commands? (See the linked devfile, the same application is run using 3000 & 3001)
3. S2I spec definition needs to be fleshed and scoped out to see if we come across any hurdles like Dockerfile spec above - https://github.com/devfile/api/pull/127/files#diff-ce0147ba3842f44db1642f6506de48a1744cce30f46475c5df580e34e36f0a59R51

[elsony]

Adding one more:
4. In dev console, the context settings in the dev console decide where the devfile is located on and the devfile location is relative to the context location. We need to figure a way to support that and represent it in a devfile.

[elsony]

Proposal: https://docs.google.com/document/d/1SL3ZJyAy8lp973EzQpY36VjTJHoyGXl2zUXokPiXwK8/edit?usp=sharing