Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create dedicated Docker Build section #14642

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@

# For more details, see https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners

/build @dockertopia @crazy-max
dockertopia marked this conversation as resolved.
Show resolved Hide resolved

/compose/ @dockertopia @usha-mandya

/compose/install.md @aevesdocker
Expand Down
13 changes: 11 additions & 2 deletions _data/toc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -1391,15 +1391,24 @@ manuals:
- path: /engine/deprecated/
title: Deprecated features

- path: /buildx/working-with-buildx/
title: Docker Buildx
- path: /engine/context/working-with-contexts/
title: Docker Context
- path: /engine/scan/
title: Docker Scan
- path: /engine/sbom/
title: Docker SBOM (Experimental)

- sectiontitle: Docker Build
section:
- path: /build/
title: Overview
- sectiontitle: Buildx component
crazy-max marked this conversation as resolved.
Show resolved Hide resolved
section:
- path: /build/buildx/
title: Working with Buildx
- path: /build/buildx/install/
title: Install Buildx

- sectiontitle: Docker Compose
section:
- path: /compose/
Expand Down
114 changes: 22 additions & 92 deletions buildx/working-with-buildx.md → build/buildx/index.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
---
title: Docker Buildx
title: Buildx component
Copy link
Contributor

@dockertopia dockertopia Jun 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Buildx is in other places referred to as the Docker CLI Buildx plugin, I think we should be consistent.


As for structure of the Build section (and page titles) here's a working proposal

It seems this will be a broad section, WDYT about structuring the content like:
Docker Build

  1. (Build) Overview
    2 About Buildx
    2.1. Installing Buildx
    2.2. About Buildx (so not to have another overview)
    2.3. Using Buildx
    2.4. Using Bake

description: Working with Docker Buildx
keywords: Docker, buildx, multi-arch
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this metadata work? Does this help search or help create related topics? Just wondering how "multi-arch" would work depending on how this metadata is picked-up.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keywords are like tags that will be used to enhance search results and are added at the bottom of the page:

image

image

They are also added as meta tag for SEO https://github.com/docker/docker.github.io/blob/9294453e5fe816824ba855e92eae3fc5b16c9c11/_includes/head.html#L39 but I don't think this is relevant anymore as it's not used by search engines anymore like Google:

[…] keyword meta tags quickly became an area where someone could stuff often-irrelevant keywords without typical visitors ever seeing those keywords. Because the keywords meta tag was so often abused, many years ago Google began disregarding the keywords meta tag.

Bing went one step further in 2011 when they announced they use the tag as a spam signal.

I think we should remove the meta keywords in the future but would need to put in place a client-side search implementation, omitting the need for the integration of third-party services (Google in our case), which might not be compliant with privacy regulations. Moreover, search even works offline, allowing users to download our documentation. I think lunr could be a good candidate for this. See for example my POC with MkDocs using a client-side search: https://crazymax.dev/docker-docs/

cc @usha-mandya @thaJeztah

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to be able to optimize the search indexes ourselves. It could take the docs experience to another level, when you'd search a term you'd already get the best matches from the team's perspective what are the best overall content candidates across types.
This means we could, if somebody searched "docker build" make sure the number 1 hit on the list is the Docker Build docs index as well as curate results across categories (refs, guides, blog, etc.).

keywords: build, buildx, buildkit
redirect_from:
- /buildx/working-with-buildx/
---

## Overview
Expand All @@ -12,79 +14,7 @@ builder toolkit. It provides the same user experience as docker build with many
new features like creating scoped builder instances and building against
multiple nodes concurrently.

## Install

### Docker Desktop

Docker Buildx is included in [Docker Desktop](../desktop/index.md) for Windows, macOS, and Linux.

### Static binaries

Docker Linux packages also include Docker Buildx when installed using the [DEB or RPM packages](../engine/install/index.md).

### Manual download

> **Important**
>
> This section is for unattended installation of the buildx component. These
> instructions are mostly suitable for testing purposes. We do not recommend
> installing buildx using manual download in production environments as they
> will not be updated automatically with security updates.
>
> On Windows, macOS, and Linux we recommend that you install [Docker Desktop](../desktop/index.md)
> instead.
{: .important}

You can also download the latest binary from the [releases page on GitHub](https://github.com/docker/buildx/releases/latest){:target="_blank" rel="noopener" class="_"}.

Rename the relevant binary and copy it to the destination matching your OS:

| OS | Binary name | Destination folder |
| -------- | -------------------- | -----------------------------------------|
| Linux | `docker-buildx` | `$HOME/.docker/cli-plugins` |
| macOS | `docker-buildx` | `$HOME/.docker/cli-plugins` |
| Windows | `docker-buildx.exe` | `%USERPROFILE%\.docker\cli-plugins` |

Or copy it into one of these folders for installing it system-wide.

On Unix environments:

* `/usr/local/lib/docker/cli-plugins` OR `/usr/local/libexec/docker/cli-plugins`
* `/usr/lib/docker/cli-plugins` OR `/usr/libexec/docker/cli-plugins`

On Windows:

* `C:\ProgramData\Docker\cli-plugins`
* `C:\Program Files\Docker\cli-plugins`

> **Note**
>
> On Unix environments, it may also be necessary to make it executable with `chmod +x`:
> ```shell
> $ chmod +x ~/.docker/cli-plugins/docker-buildx
> ```

### Dockerfile

Here is how to install and use Buildx inside a Dockerfile through the
[`docker/buildx-bin`](https://hub.docker.com/r/docker/buildx-bin) image:

```dockerfile
FROM docker
COPY --from=docker/buildx-bin:latest /buildx /usr/libexec/docker/cli-plugins/docker-buildx
RUN docker buildx version
```

## Set buildx as the default builder

Running the command [`docker buildx install`](../engine/reference/commandline/buildx_install.md)
sets up docker builder command as an alias to `docker buildx`. This results in
the ability to have [`docker build`](../engine/reference/commandline/build.md)
use the current buildx builder.

To remove this alias, run [`docker buildx uninstall`](../engine/reference/commandline/buildx_uninstall.md).

## Build with buildx
## Build with Buildx

To start a new build, run the command `docker buildx build .`

Expand Down Expand Up @@ -128,23 +58,23 @@ the shared daemon, or for isolating builds for different projects. You can creat
a new instance for a set of remote nodes, forming a build farm, and quickly
switch between them.

You can create new instances using the [`docker buildx create`](../engine/reference/commandline/buildx_create.md)
You can create new instances using the [`docker buildx create`](../../engine/reference/commandline/buildx_create.md)
command. This creates a new builder instance with a single node based on your
current configuration.

To use a remote node you can specify the `DOCKER_HOST` or the remote context name
while creating the new builder. After creating a new instance, you can manage its
lifecycle using the [`docker buildx inspect`](../engine/reference/commandline/buildx_inspect.md),
[`docker buildx stop`](../engine/reference/commandline/buildx_stop.md), and
[`docker buildx rm`](../engine/reference/commandline/buildx_rm.md) commands.
To list all available builders, use [`docker buildx ls`](../engine/reference/commandline/buildx_ls.md).
lifecycle using the [`docker buildx inspect`](../../engine/reference/commandline/buildx_inspect.md),
[`docker buildx stop`](../../engine/reference/commandline/buildx_stop.md), and
[`docker buildx rm`](../../engine/reference/commandline/buildx_rm.md) commands.
To list all available builders, use [`docker buildx ls`](../../engine/reference/commandline/buildx_ls.md).
After creating a new builder you can also append new nodes to it.

To switch between different builders, use [`docker buildx use <name>`](../engine/reference/commandline/buildx_use.md).
To switch between different builders, use [`docker buildx use <name>`](../../engine/reference/commandline/buildx_use.md).
After running this command, the build commands will automatically use this
builder.

Docker also features a [`docker context`](../engine/reference/commandline/context.md)
Docker also features a [`docker context`](../../engine/reference/commandline/context.md)
command that you can use to provide names for remote Docker API endpoints. Buildx
integrates with `docker context` to ensure all the contexts automatically get a
default builder instance. You can also set the context name as the target when
Expand All @@ -163,8 +93,8 @@ platform for the build output, (for example, `linux/amd64`, `linux/arm64`, or
When the current builder instance is backed by the `docker-container` driver,
you can specify multiple platforms together. In this case, it builds a manifest
list which contains images for all specified architectures. When you use this
image in [`docker run`](../engine/reference/commandline/run.md) or
[`docker service`](../engine/reference/commandline/service.md), Docker picks
image in [`docker run`](../../engine/reference/commandline/run.md) or
[`docker service`](../../engine/reference/commandline/service.md), Docker picks
the correct image based on the node's platform.

You can build multi-platform images using three different strategies that are
Expand All @@ -182,7 +112,7 @@ it automatically loads it through a binary registered in the `binfmt_misc`
handler.

For QEMU binaries registered with `binfmt_misc` on the host OS to work
transparently inside containers, they must be statically compiled and registered with the `fix_binary` flag.
transparently inside containers, they must be statically compiled and registered with the `fix_binary` flag.
This requires a kernel >= 4.8 and binfmt-support >= 2.1.7. You can check
for proper registration by checking if `F` is among the flags in
`/proc/sys/fs/binfmt_misc/qemu-*`. While Docker Desktop comes preconfigured
Expand All @@ -207,7 +137,7 @@ $ docker buildx create --append --name mybuild node-arm64
$ docker buildx build --platform linux/amd64,linux/arm64 .
```

Finally, depending on your project, the language that you use may have good
Finally, depending on your project, the language that you use may have good
support for cross-compilation. In that case, multi-stage builds in Dockerfiles
can be effectively used to build binaries for the platform specified with
`--platform` using the native architecture of the build node. A list of build
Expand Down Expand Up @@ -235,15 +165,15 @@ work. The build commands can be combined with general-purpose command runners
(for example, `make`). However, these tools generally invoke builds in sequence
and therefore cannot leverage the full potential of BuildKit parallelization,
or combine BuildKit’s output for the user. For this use case, we have added a
command called [`docker buildx bake`](../engine/reference/commandline/buildx_bake.md).
command called [`docker buildx bake`](../../engine/reference/commandline/buildx_bake.md).

The `bake` command supports building images from compose files, similar to
[`docker-compose build`](../compose/reference/build.md), but allowing all the
The `bake` command supports building images from compose files, similar to
[`docker-compose build`](../../compose/reference/build.md), but allowing all the
services to be built concurrently as part of a single request.

There is also support for custom build rules from HCL/JSON files allowing
better code reuse and different target groups. The design of bake is in very
early stages and we are looking for feedback from users. Let us know your
feedback by creating an issue in the
[Docker buildx](https://github.com/docker/buildx/issues){:target="_blank" rel="noopener" class="_"}
early stages and we are looking for feedback from users. Let us know your
feedback by creating an issue in the
[Docker buildx](https://github.com/docker/buildx/issues){:target="_blank" rel="noopener" class="_"}
GitHub repository.
76 changes: 76 additions & 0 deletions build/buildx/install.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
---
title: Install Docker Buildx
description: How to install Docker Buildx
keywords: build, buildx, buildkit
---

## Docker Desktop

Docker Buildx is included in [Docker Desktop](../../desktop/index.md) for Windows, macOS, and Linux.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Docker Buildx is included in [Docker Desktop](../../desktop/index.md) for Windows, macOS, and Linux.
Docker Buildx is a Docker CLI plugin and is shipped with [Docker Desktop](../../desktop/index.md). Docker Desktop is available for Windows, macOS, and Linux.


## Linux packages

Docker Linux packages also include Docker Buildx when installed using the [DEB or RPM packages](../../engine/install/index.md).

## Manual download

> **Important**
>
> This section is for unattended installation of the buildx component. These
> instructions are mostly suitable for testing purposes. We do not recommend
> installing buildx using manual download in production environments as they
> will not be updated automatically with security updates.
>
> On Windows, macOS, and Linux workstations we recommend that you install
> [Docker Desktop](../../desktop/index.md) instead. For Linux servers, we recommend
> that you follow the [instructions specific for your distribution](#linux-packages).
{: .important}

You can also download the latest binary from the [releases page on GitHub](https://github.com/docker/buildx/releases/latest){:target="_blank" rel="noopener" class="_"}.

Rename the relevant binary and copy it to the destination matching your OS:

| OS | Binary name | Destination folder |
| -------- | -------------------- | -----------------------------------------|
| Linux | `docker-buildx` | `$HOME/.docker/cli-plugins` |
| macOS | `docker-buildx` | `$HOME/.docker/cli-plugins` |
| Windows | `docker-buildx.exe` | `%USERPROFILE%\.docker\cli-plugins` |

Or copy it into one of these folders for installing it system-wide.

On Unix environments:

* `/usr/local/lib/docker/cli-plugins` OR `/usr/local/libexec/docker/cli-plugins`
* `/usr/lib/docker/cli-plugins` OR `/usr/libexec/docker/cli-plugins`

On Windows:

* `C:\ProgramData\Docker\cli-plugins`
* `C:\Program Files\Docker\cli-plugins`

> **Note**
>
> On Unix environments, it may also be necessary to make it executable with `chmod +x`:
> ```shell
> $ chmod +x ~/.docker/cli-plugins/docker-buildx
> ```

## Dockerfile

Here is how to install and use Buildx inside a Dockerfile through the
[`docker/buildx-bin`](https://hub.docker.com/r/docker/buildx-bin) image:

```dockerfile
FROM docker
COPY --from=docker/buildx-bin:latest /buildx /usr/libexec/docker/cli-plugins/docker-buildx
RUN docker buildx version
```

## Set buildx as the default builder

Running the command [`docker buildx install`](../../engine/reference/commandline/buildx_install.md)
sets up docker builder command as an alias to `docker buildx`. This results in
the ability to have [`docker build`](../../engine/reference/commandline/build.md)
use the current buildx builder.

To remove this alias, run [`docker buildx uninstall`](../../engine/reference/commandline/buildx_uninstall.md).
50 changes: 50 additions & 0 deletions build/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
---
title: Overview of Docker Build
description: Introduction and overview of Docker Build
keywords: build, buildx, buildkit
---

Docker Build is one of the most used features of the Docker Engine - users
ranging from developers, build teams, and release teams all use Docker Build.
It uses a [client-server architecture](../get-started/overview.md#docker-architecture)
that includes several tools. The most common method is to use the Docker CLI with
[`docker build` command](../engine/reference/commandline/build.md) that sends
requests to the Docker Engine that will execute your build.

Starting with version 18.09, Docker supports a new backend for executing your
builds that is provided by the [BuildKit](https://github.com/moby/buildkit)
project. The BuildKit backend provides many benefits compared to the old
implementation. As there is a new backend, there is also a new client called [Docker Buildx](buildx/index.md),
available as a CLI plugin that extends the docker command with the full
support of the features provided by BuildKit.

BuildKit is enabled by default for all users on [Docker Desktop](../desktop/index.md).
If you have installed Docker Desktop, you don't have to manually enable BuildKit.
If you have installed Docker as a [Linux package](../engine/install/index.md),
you can enable BuildKit either by using an environment variable or by making
BuildKit the default setting.

To set the BuildKit environment variable when running the
[`docker build` command](../engine/reference/commandline/build.md), run:

```console
$ DOCKER_BUILDKIT=1 docker build .
```

To enable BuildKit backend by default, set [daemon configuration](/engine/reference/commandline/dockerd/#daemon-configuration-file)
in `/etc/docker/daemon.json` feature to `true` and restart the daemon. If the
`daemon.json` file doesn't exist, create new file called `daemon.json` and then
add the following to the file:

```json
{
"features": {
"buildkit": true
}
}
```

If you're using the [`docker buildx build` command](../engine/reference/commandline/buildx_build.md),
BuildKit will always being used regardless of the environment variable or backend
configuration. See [Build with Buildx](buildx/index.md#build-with-buildx) guide
for more details.
2 changes: 1 addition & 1 deletion desktop/multi-arch.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ than just native images. Buildx accomplishes this by adding new builder
instances based on BuildKit, and leveraging Docker Desktop's technology stack
to run non-native binaries.

For more information about the Buildx CLI command, see [Buildx](../buildx/working-with-buildx.md)
For more information about the Buildx CLI command, see [Buildx](../build/buildx/index.md).
and the [`docker buildx` command line reference](../engine/reference/commandline/buildx.md).

### Build and run multi-architecture images
Expand Down
2 changes: 1 addition & 1 deletion desktop/previous-versions/2.x-mac.md
Original file line number Diff line number Diff line change
Expand Up @@ -435,7 +435,7 @@ Note that you must sign in and create a Docker ID in order to download Docker De
Docker Desktop Community 2.1.0.0 contains the following experimental features.

* Docker App: Docker App is a CLI plugin that helps configure, share, and install applications. For more information, see [Working with Docker App](/app/working-with-app/).
* Docker Buildx: Docker Buildx is a CLI plugin for extended build capabilities with BuildKit. For more information, see [Working with Docker Buildx](/buildx/working-with-buildx/).
* Docker Buildx: Docker Buildx is a CLI plugin for extended build capabilities with BuildKit. For more information, see [Buildx component](/build/buildx/).

### Bug fixes and minor changes

Expand Down
2 changes: 1 addition & 1 deletion desktop/previous-versions/2.x-windows.md
Original file line number Diff line number Diff line change
Expand Up @@ -557,7 +557,7 @@ Note that you must sign in and create a Docker ID in order to download Docker De
Docker Desktop Community 2.1.0.0 contains the following experimental features:

* Docker App: Docker App is a CLI plugin that helps configure, share, and install applications. For more information, see [Working with Docker App](/app/working-with-app/).
* Docker Buildx: Docker Buildx is a CLI plugin for extended build capabilities with BuildKit. For more information, see [Working with Docker Buildx](/buildx/working-with-buildx/).
* Docker Buildx: Docker Buildx is a CLI plugin for extended build capabilities with BuildKit. For more information, see [Buildx component](/build/buildx/).

### Bug fixes and minor changes

Expand Down
4 changes: 2 additions & 2 deletions go/buildx.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: How to install Buildx
description: Instructions on installing Buildx
keywords: Docker, buildx, multi-arch
redirect_to: /buildx/working-with-buildx/#install
keywords: Docker, Buildx, BuildKit
redirect_to: /build/buildx/install/
---