Leveraging a docker-compose file in a devfile definition

[elsony]

Use Case

As a developer working on an existing project, I have already setup my environment and built a docker-compose file which defines all the different components that I need to test my application.
I want to leverage that docker-compose in my devfile - so I can make my developer environment codified/versioned/repeatable and keep using my docker-compose as well.

Goals

It's really common for a project to get a docker-compose file. In general this is a convenient way for the developers to more easily setup their application and all its components on their local environment. It can also be used along with different containers to completely build the source code of each application components and deploy those. In this case, there might be different dockerfiles associated with each application components which will trigger a specific "task".

The goal of the effort, is to tie the docker-compose file with the devfile. So if a developer knows how to build a docker-compose or already have one, it'll get easier to onboard on devfile and leverage work and effort that has already been produced. More importantly, it would let somebody continuing to evolve the devfile and the dockercompose so they complement each others.

Examples

There are many different samples we could be looking at:

• https://www.digitalocean.com/community/tutorials/containerizing-a-node-js-application-for-development-with-docker-compose

One that might be even more interesting to look at would be this one https://github.com/RediSearch/redisearch-getting-started:

• Inside of sample-app, there is a dockercompose file
• Each application components are getting built from a defined dockerfile
• Environments variables are used to connect the services

Steps / Challenges

1- Docker-compose file with existing containers

Let's take the case that the docker-compose file is only relying on containers that are already built and available in the container catalog (docker-compose not relying on dockerfiles). We should be able to use the docker-compose file and use its services as components for the devfile.

Targets:

• We can start by having a tool, that reads the docker-compose file and convert that as a devfile.
• We would need to formalize an improvement to the specification, so that it is possible to link devfile components from the docker-compose file - so it'll not be a "convertion" of the docker-compose file to devfile, but a way to use the docker-compose and the devfile together.
• We code the API that allows to do that.

2- Docker-compose file with to-be-built containers

In this case, we need to ensure that the devfile/api will have the ability to build the containers and provide them to the developers.

Targets:

• We should look at how we can do the build of the containers on the cluster (podman, sandboxed containers... )

[elsony]

Implementation plan:

1. Docker-compose file with existing containers
   1. Create a library as part of the existing devfile library for converting a given docker-compose to existing segment(s) of the devfile model that maps the docker compose elements
      • convert something like this in a compose file to a container component:

       db:
         image: mongo:4.1.8-xenial
         container_name: db
         restart: unless-stopped
         env_file: .env
         environment:
           - MONGO_INITDB_ROOT_USERNAME=$MONGO_USERNAME
           - MONGO_INITDB_ROOT_PASSWORD=$MONGO_PASSWORD
         volumes:
           - dbdata:/data/db
         networks:
           - app-network
      

      • We will document and limit the number of elements that we support. We are not planning to support elements that do not map well to our existing devfile model, e.g. networks and restart.
      • We can refer to https://github.com/kubernetes/kompose that may help the implementation
   2. Update the merged model in the devfile library: for devfiles with the docker-compose component, convert the docker-compose component to other existing components using the library above, e.g. container components, and replace the docker-compose component with those generated components in the merged model
   3. Add a way to refer to a docker compose file from the devfile, e.g. a docker-compose component element
2. Docker-compose file with to-be-built containers
   1. Prereq: Add a way to refer to a docker compose file from the devfile, e.g. a docker-compose component element (from above)
   2. Add conversion of Dockerfile build to a dockerfile image component to the conversion function in the library (see above), e.g.

   services:
     nodejs:
       build:
         context: .
         dockerfile: Dockerfile
        image: nodejs
        container_name: nodejs
        restart: unless-stopped
        env_file: .env
        environment:
          - MONGO_USERNAME=$MONGO_USERNAME
          - MONGO_PASSWORD=$MONGO_PASSWORD
          - MONGO_HOSTNAME=db
          - MONGO_PORT=$MONGO_PORT
          - MONGO_DB=$MONGO_DB
        ports:
          - "80:8080"
        volumes:
          - .:/home/node/app
          - node_modules:/home/node/app/node_modules
        networks:
          - app-network
        command: ./wait-for.sh db:27017 -- /home/node/app/node_modules/.bin/nodemon app.js
   

   • In the library above, covert the docker-compose element As part of the outer loop support, a new image dockerfile component will be added (Introduce outerloop 'dockerfile' build & deploy #51). We can convert to that image dockerfile component
   • As part of Inner loop dockerfile support #346, it provides an inner loop container definition that allows the image to be built from a dockerfile, we'll use the same mechanism for this conversion
   • We will document and limit the number of elements that we support. We are not planning to support elements that do not map well to our existing devfile model, e.g. networks and restart.

[l0rd]

The source code of the PoC after the latest LFX mentorship round: https://github.com/vedant-kakde/kompose-DWO-poc
An issue with the PoC subtasks.

[yangcao77]

to create API spec definition : #1120
detailed work on the devfile library converter: #1121

[github-actions]

This issue is stale because it has been open for 90 days with no activity. Remove stale label or comment or this will be closed in 60 days.