-
Notifications
You must be signed in to change notification settings - Fork 8.1k
Compose Bridge early access #19702
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Compose Bridge early access #19702
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,189 @@ | ||||||
| --- | ||||||
| description: Overview of Docker Compose Bridge | ||||||
| keywords: compose, orchestration, kubernetes, bridge | ||||||
| title: Overview of Docker Compose Bridge | ||||||
| --- | ||||||
|
|
||||||
| {{< include "compose-bridge-early-access.md" >}} | ||||||
|
|
||||||
| ## Introduction | ||||||
|
|
||||||
| Docker Compose makes it easy to define a multi-container application | ||||||
| to be run on a single-node Docker Engine, relying on a `compose.yaml` file to | ||||||
| describe resources with a simple abstraction. | ||||||
|
|
||||||
| Compose Bridge lets you reuse this exact same `compose.yaml` file but | ||||||
| translate it into another platform's definition format, with a primary | ||||||
| focus on Kubernetes. This transformation can be customized to match | ||||||
| specific needs and requirements. | ||||||
|
|
||||||
| ## Usage | ||||||
|
|
||||||
| Compose Bridge is a command line tool that consumes a `compose.yaml` file | ||||||
| and runs a transformation to produce resource definitions for another platform. | ||||||
| [By default](transformation.md), it produces Kubernetes manifests and a Kustomize overlay for Docker Desktop. For example: | ||||||
| ```console | ||||||
| $ compose-bridge -f compose.yaml convert | ||||||
| Kubernetes resource api-deployment.yaml created | ||||||
| Kubernetes resource db-deployment.yaml created | ||||||
| Kubernetes resource web-deployment.yaml created | ||||||
| Kubernetes resource api-expose.yaml created | ||||||
| Kubernetes resource db-expose.yaml created | ||||||
| Kubernetes resource web-expose.yaml created | ||||||
| Kubernetes resource 0-avatars-namespace.yaml created | ||||||
| Kubernetes resource default-network-policy.yaml created | ||||||
| Kubernetes resource private-network-policy.yaml created | ||||||
| Kubernetes resource public-network-policy.yaml created | ||||||
| Kubernetes resource db-db_data-persistentVolumeClaim.yaml created | ||||||
| Kubernetes resource api-service.yaml created | ||||||
| Kubernetes resource web-service.yaml created | ||||||
| Kubernetes resource kustomization.yaml created | ||||||
| Kubernetes resource db-db_data-persistentVolumeClaim.yaml created | ||||||
| Kubernetes resource api-service.yaml created | ||||||
| Kubernetes resource web-service.yaml created | ||||||
| Kubernetes resource kustomization.yaml created | ||||||
| ``` | ||||||
|
|
||||||
| Such manifests can then be used to run the application on Kubernetes using | ||||||
| the standard deployment command `kubectl apply -k out/overlays/desktop/`. | ||||||
|
|
||||||
| ## Customization | ||||||
|
|
||||||
| The Kubernetes manifests produced by Compose Bridge | ||||||
| are designed to allow deployment on Docker Desktop with Kubernetes enabled. | ||||||
|
|
||||||
| Kubernetes is such a versatile platform that there are many ways | ||||||
| to map Compose concepts into a Kubernetes resource definitions. Compose | ||||||
| Bridge lets you customize the transformation to match your own infrastructure | ||||||
| decisions and preferences, with various level of flexibility / investment. | ||||||
|
|
||||||
|
|
||||||
| ### Modify the default templates | ||||||
|
|
||||||
| You can extract templates used by default transformation `docker/compose-bridge-kubernetes` | ||||||
| by running `compose-bridge transformations create my-template --from docker/compose-bridge-kubernetes` | ||||||
| and adjusting those to match your needs. | ||||||
|
|
||||||
| The templates will be extracted into a directory named after your template name (ie `my-template`). | ||||||
| Inside, you will find a Dockerfile that allows you to create your own image to distribute your template, as well as a directory containing the templating files. | ||||||
| You are free to edit the existing files, delete them, or [add new ones](#add-your-own-templates) to subsequently generate Kubernetes manifests that meet your needs. | ||||||
| You can then use the generated Dockerfile to package your changes into a new Transformer image, which you can then use with Compose Bridge: | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
I see 'Image transformer' used instead of 'Transformer image'
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Transformer is the concept we defined to transform your Compose model to dedicated Kubernetes resources, so here Transformer image is the right name to use |
||||||
|
|
||||||
| ```console | ||||||
| $ docker build --tag mycompany/transform --push . | ||||||
| ``` | ||||||
|
|
||||||
| You can then use your transformation as a replacement: | ||||||
| ```console | ||||||
| $ compose-bridge -f compose.yaml convert --transformation mycompany/transform | ||||||
| ``` | ||||||
|
|
||||||
| For more information, see [Templates](./templates.md). | ||||||
|
|
||||||
| ### Add your own templates | ||||||
|
|
||||||
| For resources that are not managed by Compose Bridge's default transformation, | ||||||
| you can build your own templates. The `compose.yaml` model may not offer all | ||||||
| the configuration attributes required to populate the target manifest. If this is the case, you can | ||||||
| then rely on Compose custom extensions to let developers better describe the | ||||||
| application, and offer an agnostic transformation. | ||||||
|
|
||||||
| As an illustration, if developers add `x-virtual-host` metadata | ||||||
| to service definitions in the `compose.yaml` file, you can use the following custom attribute | ||||||
| to produce Ingress rules: | ||||||
|
|
||||||
| ```yaml | ||||||
| {{ $project := .name }} | ||||||
| #! {{ $name }}-ingress.yaml | ||||||
| # Generated code, do not edit | ||||||
| apiVersion: networking.k8s.io/v1 | ||||||
| kind: Ingress | ||||||
| metadata: | ||||||
| name: virtual-host-ingress | ||||||
| namespace: {{ $project }} | ||||||
| spec: | ||||||
| rules: | ||||||
| {{ range $name, $service := .services }} | ||||||
| {{ if $service.x-virtual-host }} | ||||||
| - host: ${{ $service.x-virtual-host }} | ||||||
| http: | ||||||
| paths: | ||||||
| - path: "/" | ||||||
| backend: | ||||||
| service: | ||||||
| name: ${{ name }} | ||||||
| port: | ||||||
| number: 80 | ||||||
| {{ end }} | ||||||
| {{ end }} | ||||||
| ``` | ||||||
|
|
||||||
| Once packaged into a Docker image, you can use this custom template | ||||||
| when transforming Compose models into Kubernetes in addition to other | ||||||
| transformations: | ||||||
|
|
||||||
| ```console | ||||||
| $ compose-bridge -f compose.yaml convert \ | ||||||
| --transformation docker/compose-bridge-kubernetes \ | ||||||
| --transformation mycompany/transform | ||||||
| ``` | ||||||
|
|
||||||
| ### Build your own transformation | ||||||
|
|
||||||
| While Compose Bridge templates make it easy to customize with minimal changes, | ||||||
| you may want to make significant changes, or rely on an existing conversion tool. | ||||||
|
|
||||||
| A Compose Bridge transformation is a Docker image that is designed to get a Compose model | ||||||
| from `/in/compose.yaml` and produce platform manifests under `/out`. This simple | ||||||
| contract makes it easy to bundle an alternate transformation, as illustrated below using | ||||||
| [Kompose](https://kompose.io/): | ||||||
|
|
||||||
| ```Dockerfile | ||||||
| FROM alpine | ||||||
|
|
||||||
| # Get kompose from github release page | ||||||
| RUN apk add --no-cache curl | ||||||
| ARG VERSION=1.32.0 | ||||||
| RUN ARCH=$(uname -m | sed 's/armv7l/arm/g' | sed 's/aarch64/arm64/g' | sed 's/x86_64/amd64/g') && \ | ||||||
| curl -fsL \ | ||||||
| "https://github.com/kubernetes/kompose/releases/download/v${VERSION}/kompose-linux-${ARCH}" \ | ||||||
| -o /usr/bin/kompose | ||||||
| RUN chmod +x /usr/bin/kompose | ||||||
|
|
||||||
| CMD ["/usr/bin/kompose", "convert", "-f", "/in/compose.yaml", "--out", "/out"] | ||||||
| ``` | ||||||
|
|
||||||
| This Dockerfile bundles Kompose and defines the command to run this tool according | ||||||
| to the Compose Bridge transformation contract. | ||||||
|
|
||||||
| ## Use `compose-bridge` as a `kubectl` plugin | ||||||
|
|
||||||
| To use the `compose-bridge` binary as a `kubectl` plugin, you need to make sure that the binary is available in your PATH and the name of the binary is prefixed with `kubectl-`. | ||||||
|
|
||||||
| 1. Rename or copy the `compose-bridge` binary to `kubectl-compose_bridge`: | ||||||
|
|
||||||
| ```console | ||||||
| $ mv /path/to/compose-bridge /usr/local/bin/kubectl-compose_bridge | ||||||
| ``` | ||||||
|
|
||||||
| 2. Ensure that the binary is executable: | ||||||
|
|
||||||
| ```console | ||||||
| $ chmod +x /usr/local/bin/kubectl-compose_bridge | ||||||
| ``` | ||||||
|
|
||||||
| 3. Verify that the plugin is recognized by `kubectl`: | ||||||
|
|
||||||
| ```console | ||||||
| $ kubectl plugin list | ||||||
| ``` | ||||||
|
|
||||||
| In the output, you should see `kubectl-compose_bridge`. | ||||||
|
|
||||||
| 4. Now you can use `compose-bridge` as a `kubectl` plugin: | ||||||
|
|
||||||
| ```console | ||||||
| $ kubectl compose-bridge [command] | ||||||
| ``` | ||||||
|
|
||||||
| Replace `[command]` with any `compose-bridge` command you want to use. | ||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| --- | ||
| title: Compose Bridge templates | ||
| description: Learn about the Compose Bridge templates syntax | ||
| keywords: compose, bridge, templates | ||
| --- | ||
|
|
||
| {{< include "compose-bridge-early-access.md" >}} | ||
|
|
||
| Compose Bridge's default transformation uses templates to produce Kubernetes manifests. | ||
| This page describes the templating mechanism. | ||
|
|
||
| ## Syntax | ||
|
|
||
| Templates are plain text files, using [go-template](https://pkg.go.dev/text/template) | ||
| to allow logic and data injection based on the `compose.yaml` model. | ||
|
|
||
| When a template is executed, it must produce a YAML file. Multiple files can be generated | ||
| as long as those are separated by `---` | ||
|
|
||
| The first line, when creating the YAML file, defines the file being generated using a custom notation: | ||
| ```yaml | ||
| #! manifest.yaml | ||
| ``` | ||
| With this header comment, `manifest.yaml` will be created by Compose Bridge with the content following | ||
| the annotation. | ||
|
|
||
| Combining these together, you can write a template to iterate over some of Compose resources, | ||
| then for each resource you can produce a dedicated manifest: | ||
|
|
||
| ```yaml | ||
| {{ range $name, $service := .services }} | ||
| --- | ||
| #! {{ $name }}-manifest.yaml | ||
| # Generated code, do not edit | ||
| key: value | ||
| ## ... | ||
| {{ end }} | ||
| ``` | ||
|
|
||
| This example produces a manifest file for each and every Compose service in you Compose model. | ||
|
|
||
|
|
||
| ## Input | ||
|
|
||
| The input compose model is the canonical yaml model you can get by running | ||
| `docker compose config`. Within a template you can access model nodes using | ||
| dot notation: | ||
|
|
||
| ```yaml | ||
| # iterate over a yaml sequence | ||
| {{ range $name, $service := .services }} | ||
| # access a nested attribute using dot notation | ||
| {{ if eq $service.deploy.mode "global" }} | ||
| kind: DaemonSet | ||
| {{ end }} | ||
| {{ end }} | ||
| ``` | ||
|
|
||
| You can check the [Compose Specification json-spec file](https://github.com/compose-spec/compose-go/blob/main/schema/compose-spec.json) to have a full overview of the Compose model. | ||
|
|
||
| ## Helpers | ||
|
|
||
| As part of the Go templating syntax, Compose Bridge offers a set of helper functions: | ||
|
|
||
| - `seconds`: convert a [duration](https://github.com/compose-spec/compose-spec/blob/master/11-extension.md#specifying-durations) into an integer | ||
| - `uppercase` convert a string into upper case characters | ||
| - `title`: convert a string by capitalizing first letter of each word | ||
| - `safe`: convert a string into a safe identifier, replacing all characters but \[a-z\] with `-` | ||
| - `truncate`: removes the N first elements from a list | ||
| - `join`: group elements from a list into a single string, using a separator | ||
| - `base64`: encode string as base64 | ||
| - `map`: transform value according to mappings expressed as `"value -> newValue"` strings | ||
| - `indent`: writes string content indented by N spaces | ||
| - `helmValue`: write the string content as a template value in final file |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| --- | ||
| title: Compose Bridge default transformation | ||
| description: Learn about the Compose Bridge default transformation | ||
| keywords: compose, bridge, kubernetes | ||
| --- | ||
|
|
||
| {{< include "compose-bridge-early-access.md" >}} | ||
|
|
||
| Compose Bridge produces Kubernetes manifests so you can deploy | ||
| your Compose application to Kubernetes that is enabled on Docker Desktop. | ||
|
|
||
| Based on an arbitrary `compose.yaml` project, Compose Bridge will produce: | ||
|
|
||
| - A [Namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) so all your resources are isolated and don't colide with another deployment | ||
| - A [ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) with an entry for each and every config resource in your Compose model | ||
| - [Deployments](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) for application services | ||
| - [Services](https://kubernetes.io/docs/concepts/services-networking/service/) for ports exposed by your services, used for service-to-service communication | ||
| - [Services](https://kubernetes.io/docs/concepts/services-networking/service/) for ports published by your services, with type `LoadBalancer` so that Docker Desktop will also expose same port on host | ||
| - [Network policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) to replicate the networking topology expressed in Compose | ||
| - [PersistentVolumeClaims](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) for your volumes, using `hostpath` storage class so that Docker Desktop manages volume creation | ||
| - [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) with your secret encoded - this is designed for local use in a testing environment | ||
|
|
||
| And a Kustomize overlay dedicated to Docker Desktop with: | ||
| - Loadbalancer for services which need to expose ports on host | ||
| - A PersistentVolumeClaim to use the Docker Desktop storage provisioner `desktop-storage-provisioner` | ||
| - A Kustomize file to link the all those resources together |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| > **Early Access** | ||
| > | ||
| > Compose Bridge command line is an [early access](/release-lifecycle#early-access-ea) product. | ||
| > | ||
| { .restricted } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.