diff --git a/content/agents/claude-code/quickstart.mdx b/content/agents/claude-code/quickstart.mdx index b85ad4e..1bc073f 100644 --- a/content/agents/claude-code/quickstart.mdx +++ b/content/agents/claude-code/quickstart.mdx @@ -122,13 +122,23 @@ This command tells the Depot control plane to start a new agent sandbox for Clau And that's it! Your Depot organization is set up to use remote agent sandboxes for Claude Code. +## Manage sessions using the Depot dashboard + +In addition to using the CLI, you can also manage your remote agent sandboxes directly from the Depot dashboard: + +1. Log in to your [Depot dashboard](/orgs). +2. To view all your sessions, click **Claude Code**. +3. From this view, you can: + - **Resume existing sessions**: Click on any session to view its details, then use the prompt input at the bottom to resume the session with a new message. + - **Start new sessions**: Click the **New sandbox** button to launch a fresh Claude Code session in a new remote agent sandbox. You can select a repository, branch, and provide an initial prompt. + ## Next steps Try the following with your remote agent sandbox: - Work with different Git repositories that your Git credentials or Depot Code app have access to. - Switch between branches using the `--branch` flag. -- Resume a session using the `--resume` flag. +- Resume a session using the `--resume` flag or via the Depot dashboard. - Fork a new session from an existing session using the `--resume` and `--fork-session` flags together. Run `depot claude --help` or check the [CLI reference](/docs/cli/reference#depot-claude) to see all the available command options. diff --git a/content/agents/overview.mdx b/content/agents/overview.mdx index 6dcac50..0c77385 100644 --- a/content/agents/overview.mdx +++ b/content/agents/overview.mdx @@ -49,7 +49,12 @@ Agent sandboxes run on Depot's optimized infrastructure with plans to provide au ### Web UI for sessions -Easily manage your agent sessions through the Depot web UI. Start, resume, and share sessions with just a few clicks. The web interface provides a simple way to monitor your active sessions and access session logs. +Manage your agent sessions through the Depot dashboard: + +- **Session overview**: View all your Claude Code sessions with their status, last updated time, and whether they used a remote agent +- **Resume sessions**: Pick up exactly where you left off by resuming any session with a new prompt +- **Start new sessions**: Launch fresh sandboxes by entering a prompt and selecting a repository +- **Session details**: View the session details, full conversation, and sandbox execution history ## How it works @@ -76,7 +81,9 @@ Behind the scenes, Depot will do all of the following: 5. **Session saving**: When Claude Code has finished it's work and exits, the session state is preserved for later resumption 6. **Easy resumption**: Use `--resume ` to continue from any environment -Here is an example workflow where we start a new session with one prompt, then resume it later to continue working: +The following examples start a new session with one prompt, then resume it later to continue working. + +**Using the Depot CLI:** ```shell # Start a new session with a custom ID @@ -89,6 +96,12 @@ depot claude \ depot claude --resume feature-auth "This looks good, but we need to add the concept of a user profile now." ``` +**Using the Depot dashboard:** + +1. Navigate to the [**Claude Code** section](https://depot.dev/orgs/_/claude) in your dashboard +2. Click **New sandbox** to start a fresh session, selecting your repository and providing an initial prompt +3. Later, click on any session to view its details and use the prompt input to resume it with additional instructions + ## Pricing Depot remote agent sandboxes are available on **all plans** and are billed at a usage rate of **$0.01/minute** with no included usage for remote agents. diff --git a/content/api/api-container-builds-tutorial.mdx b/content/api/api-container-builds-tutorial.mdx new file mode 100644 index 0000000..efd507e --- /dev/null +++ b/content/api/api-container-builds-tutorial.mdx @@ -0,0 +1,539 @@ +--- +title: 'Container builds API tutorial' +--- + +This tutorial walks you through using Depot API to build Docker images programmatically. The container builds API allows you to build Docker images on behalf of your users without managing build infrastructure. + +Depot provides two SDKs for building images via the API: + +**Node.js SDK + Depot CLI** + +The Node.js SDK handles project management and build registration, then delegates the actual build to the Depot CLI. This approach is simpler and requires less code. + +**Go SDK + BuildKit** + +The Go SDK provides direct access to BuildKit, giving you full control over the build process. You manage the connection, configuration, and build steps yourself. + +--- + +## Choose your approach + +Select the SDK that best fits your use case: + +
+ + Node.js SDK + Depot CLI + + + + +
+ +## Prerequisites + +- A Depot account with an organization +- Node.js installed locally +- [Depot CLI](/docs/cli/installation) installed + +## Setup + +This tutorial uses code from our [example repository](https://github.com/depot/examples/tree/main/build-api). Clone it to follow along: + +```shell +git clone https://github.com/depot/examples.git +cd examples/build-api +``` + +The example repository contains the following Node.js examples under (`nodejs/`): + +- [`list-projects.js`](https://github.com/depot/examples/blob/main/build-api/nodejs/src/list-projects.js) - List all projects +- [`create-project.js`](https://github.com/depot/examples/blob/main/build-api/nodejs/src/create-project.js) - Create a new project +- [`delete-project.js`](https://github.com/depot/examples/blob/main/build-api/nodejs/src/delete-project.js) - Delete a project +- [`create-build.js`](https://github.com/depot/examples/blob/main/build-api/nodejs/src/create-build.js) - Build image with options (load/save/push) + +To get started, install Node.js dependencies: + +```bash +cd nodejs +npm install +``` + +## Step 1: Create an organization token + +1. Navigate to your organization settings in the Depot dashboard +2. Scroll to **API Tokens** section +3. Enter a description (e.g., `test-token`) and click **Create token** +4. Copy the token and save it securely (you won't see it again) + +Set the token as an environment variable: + +```shell +export DEPOT_TOKEN= +``` + +## Step 2: Install Depot CLI + +Install via curl: + +```shell +curl -L https://depot.dev/install-cli.sh | sh +``` + +Or via Homebrew (macOS): + +```shell +brew install depot/tap/depot +``` + +## Step 3: Create a project + +Projects in Depot provide isolated builder infrastructure and cache storage. We recommend creating a separate project for each customer organization to maximize cache effectiveness and prevent cache poisoning. + +To create a project, use the `ProjectService.createProject` method with your organization token: + +```javascript +const {depot} = require('@depot/sdk-node') + +const headers = { + Authorization: `Bearer ${process.env.DEPOT_TOKEN}`, +} + +const result = await depot.core.v1.ProjectService.createProject( + { + name: 'my-project', + regionId: 'us-east-1', + cachePolicy: {keepBytes: 50 * 1024 * 1024 * 1024, keepDays: 14}, // 50GB, 14 days + }, + {headers}, +) + +console.log(result.project.projectId) +``` + +Try it with the example: `node nodejs/src/create-project.js my-project` + +Save the `projectId` from the output, you'll need it for builds. + +Example output: + +```text +_Project { + projectId: 'krt0wtn195', + organizationId: '3d1h48dqlh', + name: 'my-project', + regionId: 'us-east-1', + createdAt: Timestamp { seconds: 1708021346n, nanos: 83000000 }, + cachePolicy: _CachePolicy { keepBytes: 53687091200n, keepDays: 14 } +} +``` + +## Step 4: Build a Docker image + +To build an image, first register a build with the Build API using `BuildService.createBuild`. This returns a build ID and one-time build token that you pass to the Depot CLI: + +```javascript +const {depot} = require('@depot/sdk-node') +const {exec} = require('child_process') + +const headers = { + Authorization: `Bearer ${process.env.DEPOT_TOKEN}`, +} + +// Register the build +const result = await depot.build.v1.BuildService.createBuild({projectId: ''}, {headers}) + +// Execute build with Depot CLI +exec( + 'depot build --load .', + { + env: { + DEPOT_PROJECT_ID: '', + DEPOT_BUILD_ID: result.buildId, + DEPOT_TOKEN: result.buildToken, + }, + }, + (error, stdout, stderr) => { + if (error) { + console.error(`Error: ${error}`) + return + } + console.log(stdout) + }, +) +``` + +Try it with the example: `node nodejs/src/create-build.js ` + +The `--load` flag downloads the built image to your local Docker daemon. + +## Step 5: Run the container + +List your local Docker images: + +```shell +docker image ls +``` + +Run the built container: + +```shell +docker run +``` + +You should see "Hello World" output from the Node.js application. + +## Step 6: Save to a registry + +### Push to Depot Registry + +Instead of loading locally with `--load`, you can save the image to Depot Registry using the `--save` flag: + +```javascript +exec('depot build --save .', { + env: { + DEPOT_PROJECT_ID: '', + DEPOT_BUILD_ID: result.buildId, + DEPOT_TOKEN: result.buildToken, + }, +}) +``` + +Try it: `node nodejs/src/create-build.js save` + +The build output shows how to pull or push the saved image: + +```text +Saved target: + To pull: depot pull --project + To push: depot push --project --tag +``` + +### Push to external registries + +To push directly to Docker Hub, GHCR, ECR, or other registries during the build, use the `--push` flag with `--tag`: + +```javascript +exec('depot build --push --tag docker.io/myuser/myapp:latest .', { + env: { + DEPOT_PROJECT_ID: '', + DEPOT_BUILD_ID: result.buildId, + DEPOT_TOKEN: result.buildToken, + }, +}) +``` + +First authenticate with `docker login`, then pushing to other registries simply requires setting the proper image name: + +```shell +# Docker Hub +node nodejs/src/create-build.js push docker.io/myuser/myapp:latest + +# GitHub Container Registry +node nodejs/src/create-build.js push ghcr.io/myorg/myapp:latest + +# AWS ECR +node nodejs/src/create-build.js push 123456789012.dkr.ecr.us-east-1.amazonaws.com/myapp:latest +``` + +
+
+ +
+ + Go SDK + BuildKit + + + + +
+ +## Prerequisites + +- A Depot account with an organization +- Go 1.21+ installed locally + +## Setup + +This tutorial uses code from our [example repository](https://github.com/depot/examples/tree/main/build-api). Clone it to follow along: + +```shell +git clone https://github.com/depot/examples.git +cd examples/build-api +``` + +The Go examples use two packages: + +- **Buf Connect API** (`buf.build/gen/go/depot/api`) - For project management +- **Depot Go SDK** (`github.com/depot/depot-go`) - For builds + +Available examples: + +- [`list-projects/main.go`](https://github.com/depot/examples/blob/main/build-api/go/list-projects/main.go) - List all projects +- [`create-project/main.go`](https://github.com/depot/examples/blob/main/build-api/go/create-project/main.go) - Create a new project +- [`delete-project/main.go`](https://github.com/depot/examples/blob/main/build-api/go/delete-project/main.go) - Delete a project +- [`create-build/main.go`](https://github.com/depot/examples/blob/main/build-api/go/create-build/main.go) - Build image (saved to Depot) +- [`build-and-push/main.go`](https://github.com/depot/examples/blob/main/build-api/go/build-and-push/main.go) - Build and push to external registry + +Install dependencies: + +```bash +cd go +go mod download +``` + +## Build flow overview + +Building with the Go SDK involves three steps: + +1. **Register a build** - Request a build from the Depot API +2. **Acquire a builder machine** - Get an ephemeral BuildKit machine with your project cache +3. **Build and push** - Connect to BuildKit and execute the build + +See the complete implementation in [`build-and-push/main.go`](https://github.com/depot/examples/blob/main/build-api/go/build-and-push/main.go). + +## Step 1: Create an organization token + +1. Navigate to your organization settings in the Depot dashboard +2. Scroll to **API Tokens** section +3. Enter a description (e.g., `test-token`) and click **Create token** +4. Copy the token and save it securely (you won't see it again) + +Set the token as an environment variable: + +```shell +export DEPOT_TOKEN= +``` + +## Step 2: Create a project + +Projects in Depot provide isolated builder infrastructure and cache storage. To create a project, use the Buf Connect API client with `ProjectService.CreateProject`: + +```go +import ( + "net/http" + corev1 "buf.build/gen/go/depot/api/protocolbuffers/go/depot/core/v1" + "buf.build/gen/go/depot/api/connectrpc/go/depot/core/v1/corev1connect" + "connectrpc.com/connect" +) + +token := os.Getenv("DEPOT_TOKEN") + +// Create the Project Service client +client := corev1connect.NewProjectServiceClient( + http.DefaultClient, + "https://api.depot.dev", +) + +// Create a new project +req := connect.NewRequest(&corev1.CreateProjectRequest{ + Name: "my-project", + RegionId: "us-east-1", + CachePolicy: &corev1.CachePolicy{ + KeepGb: 50, // 50GB + KeepDays: 14, // 14 days + }, +}) + +// Add authentication header +req.Header().Set("Authorization", fmt.Sprintf("Bearer %s", token)) + +resp, err := client.CreateProject(ctx, req) +if err != nil { + log.Fatal(err) +} + +log.Printf("Project ID: %s", resp.Msg.Project.ProjectId) +``` + +Try it with the example: `go run ./create-project/main.go my-project` + +Save the project ID, you'll need it for builds. + +## Step 3: Register a build + +To start a build, register it with the Build API using `build.NewBuild`. This returns a build ID and one-time build token: + +```go +import ( + "github.com/depot/depot-go/build" + cliv1 "github.com/depot/depot-go/proto/depot/cli/v1" +) + +token := os.Getenv("DEPOT_TOKEN") +projectID := os.Getenv("DEPOT_PROJECT_ID") + +build, err := build.NewBuild(ctx, &cliv1.CreateBuildRequest{ + ProjectId: projectID, +}, token) +if err != nil { + log.Fatal(err) +} + +// Report build result when finished +var buildErr error +defer build.Finish(buildErr) +``` + +The `build.Finish()` call reports success or failure back to Depot when your build completes. + +## Step 4: Acquire a builder machine + +With your build registered, acquire an ephemeral BuildKit machine using `machine.Acquire`. The machine comes pre-configured with your project's cache: + +```go +import "github.com/depot/depot-go/machine" + +buildkit, buildErr := machine.Acquire(ctx, build.ID, build.Token, "arm64") +if buildErr != nil { + return +} +defer buildkit.Release() +``` + +Specify `"arm64"` or `"amd64"` for your target platform. Released machines stay alive for 2 minutes to serve subsequent builds. + +## Step 5: Connect to BuildKit + +Connect to your BuildKit machine using `buildkit.Connect`: + +```go +import "github.com/moby/buildkit/client" + +buildkitClient, buildErr := buildkit.Connect(ctx) +if buildErr != nil { + return +} +``` + +This establishes a secure mTLS connection to the BuildKit endpoint. + +## Step 6: Configure the build + +Configure your build by creating a `SolveOpt` with your Dockerfile path, build context, and export settings: + +```go +import ( + "github.com/docker/cli/cli/config" + "github.com/moby/buildkit/session" + "github.com/moby/buildkit/session/auth/authprovider" +) + +solverOptions := client.SolveOpt{ + Frontend: "dockerfile.v0", + FrontendAttrs: map[string]string{ + "filename": "Dockerfile", + "platform": "linux/arm64", + }, + LocalDirs: map[string]string{ + "dockerfile": ".", + "context": ".", + }, + Exports: []client.ExportEntry{ + { + Type: "image", + Attrs: map[string]string{ + "name": "myuser/myapp:latest", + "oci-mediatypes": "true", + "push": "true", + }, + }, + }, + Session: []session.Attachable{ + authprovider.NewDockerAuthProvider(config.LoadDefaultConfigFile(os.Stderr), nil), + }, +} +``` + +The `Session` uses your Docker credentials from `docker login` to authenticate registry pushes. + +## Step 7: Stream build output (optional) + +To monitor build progress, create a status channel and process BuildKit status messages: + +```go +import "encoding/json" + +buildStatusCh := make(chan *client.SolveStatus, 10) +go func() { + enc := json.NewEncoder(os.Stdout) + enc.SetIndent("", " ") + for status := range buildStatusCh { + _ = enc.Encode(status) + } +}() +``` + +This streams build progress in real-time as JSON. + +## Step 8: Build and push + +Execute the build with `buildkitClient.Solve`. BuildKit automatically reuses cached layers from your project: + +```go +_, buildErr = buildkitClient.Solve(ctx, nil, solverOptions, buildStatusCh) +if buildErr != nil { + return +} +``` + +When complete, your image is pushed to the registry specified in the `Exports` configuration. + +Try the complete example: `DEPOT_PROJECT_ID= go run ./build-and-push/main.go` + +### Push to third-party registries + +To push to external registries, configure the full registry path in your image name and provide authentication. + +#### Set the full registry path: + +```go +Exports: []client.ExportEntry{ + { + Type: "image", + Attrs: map[string]string{ + "name": "docker.io/myuser/myapp:latest", // or ghcr.io, ECR, etc. + "oci-mediatypes": "true", + "push": "true", + }, + }, +}, +``` + +The `build-and-push` example supports two options for authentication: + +#### Option 1: Docker login credentials (default) + +After running `docker login`, BuildKit automatically uses credentials from `~/.docker/config.json`: + +```bash +docker login docker.io +DEPOT_PROJECT_ID= go run ./build-and-push/main.go docker.io/user/app:latest +``` + +#### Option 2: Programmatic credentials (for CI/CD) + +Provide credentials via environment variables: + +```bash +DEPOT_PROJECT_ID= \ +REGISTRY_USERNAME=myuser \ +REGISTRY_PASSWORD=mytoken \ +REGISTRY_URL=https://index.docker.io/v1/ \ +go run ./build-and-push/main.go docker.io/user/app:latest +``` + +The example automatically detects which method to use based on the presence of `REGISTRY_USERNAME` and `REGISTRY_PASSWORD`. + +See the complete working examples in the repository: [`go/create-build/main.go`](https://github.com/depot/examples/blob/main/build-api/go/create-build/main.go) and [`go/build-and-push/main.go`](https://github.com/depot/examples/blob/main/build-api/go/build-and-push/main.go). + +
+
+ +--- + +## Next steps + +- Review the [API reference](/docs/api/overview) for complete API documentation +- Explore the [Node.js SDK on GitHub](https://github.com/depot/sdk-node) +- Explore the [Go SDK on GitHub](https://github.com/depot/depot-go) +- Learn about [BuildKit in depth](/blog/buildkit-in-depth) diff --git a/content/container-builds/reference/api-authentication.mdx b/content/api/authentication.mdx similarity index 100% rename from content/container-builds/reference/api-authentication.mdx rename to content/api/authentication.mdx diff --git a/content/container-builds/reference/api-overview.mdx b/content/api/overview.mdx similarity index 99% rename from content/container-builds/reference/api-overview.mdx rename to content/api/overview.mdx index 86b92ff..3f3fc3d 100644 --- a/content/container-builds/reference/api-overview.mdx +++ b/content/api/overview.mdx @@ -13,7 +13,7 @@ Our API is built with Connect, offering [multiprotocol support](https://connectr ## Authentication -Authentication to the API is handled via an `Authorization` header with the value being an Organization Token that you generate inside of your Organization Settings. See the [Authentication docs](/docs/container-builds/reference/api-authentication) for more details. +Authentication to the API is handled via an `Authorization` header with the value being an Organization Token that you generate inside of your Organization Settings. See the [Authentication docs](/docs/api/authentication) for more details. ## Security diff --git a/content/cache/overview.mdx b/content/cache/overview.mdx index 89e6767..ef1530d 100644 --- a/content/cache/overview.mdx +++ b/content/cache/overview.mdx @@ -1,7 +1,7 @@ --- title: Depot Cache ogTitle: Overview of Depot remote caching -description: Learn how to use Depot remote cache for exponentially faster builds for tools like Bazel, Go, Turborepo, sccache, Pants, and Gradle. +description: Learn how to use Depot remote cache for exponentially faster builds for tools like GitHub Actions, Bazel, Go, Turborepo, sccache, Pants, and Gradle. --- import {CacheToolLogoGrid} from '~/components/docs/CacheToolLogoGrid' @@ -12,7 +12,7 @@ One of the biggest benefits of adopting advanced build tools like Bazel is the a ## Supported tools -Depot Cache integrates with build tools that support remote caching like Bazel, Go, Turborepo, sccache, Pants, and Gradle. For information about how to configure each tool to use Depot Cache, see the tool documentation: +Depot Cache integrates with build tools that support remote caching like GitHub Actions, Bazel, Go, Turborepo, sccache, Pants, and Gradle. For information about how to configure each tool to use Depot Cache, see the tool documentation: diff --git a/content/cache/reference/bazel.mdx b/content/cache/reference/bazel.mdx index c8039bd..4bfbbae 100644 --- a/content/cache/reference/bazel.mdx +++ b/content/cache/reference/bazel.mdx @@ -10,30 +10,13 @@ description: Learn how to use Depot remote caching for Bazel builds ## Configuring Bazel to use Depot Cache -Depot Cache can be used with Bazel from Depot's managed GitHub Actions runners, from your local machine, or from any CI/CD system. +Depot Cache can be used with Bazel from Depot's managed GitHub Actions runners, from your local machine, from any CI/CD system, or within containerized builds using Dockerfiles or Bake files. ### From Depot-managed Actions runners [Depot GitHub Actions runners](/docs/github-actions/overview) are pre-configured to use Depot Cache with Bazel - each runner is launched with a `$HOME/.bazelrc` file that is pre-populated with the connection details for Depot Cache. -If this automatic configuration is incompatible with your specific setup, you can disable automatic configuration in your organization settings page and manually configure Bazel to use Depot Cache as described below. - -### From your local machine or any CI/CD system - -To manually configure Bazel to use Depot Cache, you will need to set two build flags in your `.bazelrc` file. Configure Bazel to use the Depot Cache service endpoint and set API token as the `authorization` header: - -```bash -build --remote_cache=https://cache.depot.dev -build --remote_header=authorization=DEPOT_TOKEN -``` - -If you are a member of multiple organizations, and you are authenticating with a user token, you must additionally specify which organization to use for cache storage with the `x-depot-org` header: - -```bash -build --remote_header=x-depot-org=DEPOT_ORG_ID -``` - -Once Bazel is configured to use Depot Cache, you can then run your builds as you normally would. Bazel will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. +If you don't want Depot to override the `$HOME/.bazelrc` file on each runner, disable **Allow Actions jobs to automatically connect to Depot Cache** in your organization settings page. You can manually configure Bazel to use Depot Cache as described in the "Using Depot Cache from your local machine or any CI/CD system" section. ### Using Depot Cache with Bazel in `depot/build-push-action` @@ -74,3 +57,94 @@ RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ ``` Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +### Using Depot Cache from your local machine or any CI/CD system + +To manually configure Bazel to use Depot Cache, you will need to set two build flags in your `.bazelrc` file. Configure Bazel to use the Depot Cache service endpoint and set API token as the `authorization` header: + +```bash +build --remote_cache=https://cache.depot.dev +build --remote_header=authorization=DEPOT_TOKEN +``` + +If you are a member of multiple organizations, and you are authenticating with a user token, you must additionally specify which organization to use for cache storage with the `x-depot-org` header: + +```bash +build --remote_header=x-depot-org=DEPOT_ORG_ID +``` + +After Bazel is configured to use Depot Cache, you can then run your builds as you normally would. Bazel will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. + +### Using Depot Cache with Bazel in Depot CLI + +When building directly with Depot CLI, follow these steps: + +1. Update your Dockerfile to mount the secret and configure Bazel: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Create .bazelrc with cache configuration +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + echo "build --remote_cache=https://cache.depot.dev" >> ~/.bazelrc && \ + echo "build --remote_header=authorization=${DEPOT_TOKEN}" >> ~/.bazelrc && \ + bazel build +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +2. Build with Depot CLI: + +```shell +depot build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +Or with Docker Buildx: + +```shell +docker buildx build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +### Using Depot Cache with Bazel in Bake files + +When using Bake files to build Docker images containing Bazel workspaces, you can pass secrets through the `target.secret` attribute: + +1. Define the secret in your `docker-bake.hcl` file: + +```hcl +target "default" { + context = "." + dockerfile = "Dockerfile" + tags = ["your-image:tag"] + secret = [ + { + type = "env" + id = "DEPOT_TOKEN" + } + ] +} +``` + +2. Update your Dockerfile to mount the secret and configure Bazel: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Create .bazelrc with cache configuration +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + echo "build --remote_cache=https://cache.depot.dev" >> ~/.bazelrc && \ + echo "build --remote_header=authorization=${DEPOT_TOKEN}" >> ~/.bazelrc && \ + bazel build +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +3. Run the build with `depot bake`: + +```shell +DEPOT_TOKEN=your_token depot bake +``` diff --git a/content/cache/reference/github-actions.mdx b/content/cache/reference/github-actions.mdx new file mode 100644 index 0000000..e382237 --- /dev/null +++ b/content/cache/reference/github-actions.mdx @@ -0,0 +1,62 @@ +--- +title: GitHub Actions +ogTitle: Remote caching for GitHub Actions workflows +description: Learn how to use Depot remote caching for GitHub Actions workflows +--- + +[**Depot Cache**](/docs/cache/overview) provides a remote cache service that can be used with GitHub Actions cache when running workflows on Depot-managed GitHub Actions runners, which provides faster upload and download speeds compared to the standard GitHub Actions cache. + +## Using GitHub Actions cache with Depot Cache + +Depot Cache for GitHub Actions is only available when using [Depot GitHub Actions runners](/docs/github-actions/overview). Standard GitHub-hosted runners or self-hosted runners will continue to use the standard GitHub Actions cache. + +### From Depot-managed Actions runners + +[Depot GitHub Actions runners](/docs/github-actions/overview) are pre-configured to use Depot Cache for all GitHub Actions cache operations. When running workflows on Depot runners, any action that uses the GitHub Actions cache API will automatically use Depot Cache instead of the standard GitHub Actions cache. + +This includes: + +- `actions/cache` - The standard caching action +- `actions/setup-node` - Caches npm/yarn dependencies when cache option is enabled +- `actions/setup-python` - Caches pip dependencies when cache option is enabled +- `actions/setup-java` - Caches Maven/Gradle dependencies when cache option is enabled +- Any other action that uses the GitHub Actions cache API + +#### Example workflow + +A workflow using `actions/cache` will automatically use Depot Cache when running on Depot runners, with no changes to your workflow file: + +```yaml +steps: + - uses: actions/checkout@v4 + + - name: Cache dependencies + uses: actions/cache@v4 + with: + path: ~/.npm + key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }} + restore-keys: | + ${{ runner.os }}-node- + + - name: Install dependencies + run: npm ci +``` + +## Cache behavior + +### Repository scoping + +Cache entries stored in Depot Cache are scoped by repository. This means that cache entries can be read only by the same repository that saved them. + +Scoping cache by repository has the following benefits: + +- Repositories don't have key collisions when using the same cache key +- One repository can't unexpectedly read cache entries from another repository of a different trust level (for example, a public repository reading from or writing to a private repository) + +### Branch isolation + +Depot Cache doesn't enforce cache isolation based on the branch the job is run from. Whether the branch is main or another topic branch, the cache is contributed to the same namespace making it accessible to other jobs. This allows you to be in control of your cache isolation based on how you format your cache keys. + +### Cache retention and limits + +Depot Cache for GitHub Actions is billed at $0.20 per GB of usage. You can configure cache retention periods (7, 14, or 30 days) and size limits (25GB to 500GB, or no limit) in your organization settings. The default retention period is 14 days. diff --git a/content/cache/reference/gocache.mdx b/content/cache/reference/gocache.mdx index eaed62e..46bebdd 100644 --- a/content/cache/reference/gocache.mdx +++ b/content/cache/reference/gocache.mdx @@ -6,15 +6,58 @@ description: Learn how to use Depot remote caching for Go ## Configuring Go to use Depot Cache -Depot Cache can be used with Go from Depot's managed GitHub Actions runners, from your local machine, or from any CI/CD system. +Depot Cache can be used with Go from Depot's managed GitHub Actions runners, from your local machine, from any CI/CD system, or within containerized builds using Dockerfiles or Bake files. ### From Depot-managed Actions runners [Depot GitHub Actions runners](/docs/github-actions/overview) are pre-configured to use Depot Cache with Go - each runner is launched with the `GOCACHEPROG` environment variable pre-populated with the connection details for Depot Cache. -If this automatic configuration is incompatible with your specific setup, you can disable automatic configuration in your organization settings page and manually configure `GOCACHEPROG` to use Depot Cache as described below. +If you don't want Depot to set up the `GOCACHEPROG` environment variable on each runner, disable **Allow Actions jobs to automatically connect to Depot Cache** in your organization settings page. You can manually configure `GOCACHEPROG` to use Depot Cache as described in the "Using Depot Cache from your local machine or any CI/CD system" section. -### From your local machine or any CI/CD system +### Using Depot Cache with Go in `depot/build-push-action` + +When using `depot/build-push-action` to build Docker images that contain Go projects, your build needs access to Go's remote cache credentials to benefit from caching. + +These credentials are not automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners (which have automatic access to Depot Cache environment variables), containerized builds execute in isolated VMs that require explicit configuration. + +Follow these steps to securely pass your Go cache credentials into your Docker build: + +1. Store the Depot token in a GitHub Secret named `DEPOT_TOKEN`. + +2. Configure your GitHub Action to pass secrets to the container build: + +```yaml +- name: Build and push + uses: depot/build-push-action@v1 + with: + context: . + file: ./Dockerfile + push: true + tags: your-image:tag + secrets: | + "DEPOT_TOKEN=${{ secrets.DEPOT_TOKEN }}" +``` + +3. Update your Dockerfile to install the Depot CLI and configure Go cache: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Install Depot CLI +RUN curl -L https://depot.dev/install-cli.sh | sh + +# Mount secret and set GOCACHEPROG +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + PATH="/root/.depot/bin:$PATH" \ + GOCACHEPROG="depot gocache" \ + go build -v ./ +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +### Using Depot Cache from your local machine or any CI/CD system To manually configure Go to use Depot Cache, set the `GOCACHEPROG` in your environment: @@ -42,33 +85,64 @@ To set verbose output, add the --verbose option: export GOCACHEPROG='depot gocache --verbose' ``` -Once Go is configured to use Depot Cache, you can then run your builds as you normally would. Go will automatically communicate with `GOCACHEPROG` to fetch from Depot Cache and reuse any stored build artifacts from your previous builds. +After Go is configured to use Depot Cache, you can then run your builds as you normally would. Go will automatically communicate with `GOCACHEPROG` to fetch from Depot Cache and reuse any stored build artifacts from your previous builds. -### Using Depot Cache with Go in `depot/build-push-action` +### Using Depot Cache with Go in Depot CLI -When using `depot/build-push-action` to build Docker images that contain Go projects, your build needs access to Go's remote cache credentials to benefit from caching. +When building directly with Depot CLI, follow these steps: -These credentials are not automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners (which have automatic access to Depot Cache environment variables), containerized builds execute in isolated VMs that require explicit configuration. +1. Update your Dockerfile to install the Depot CLI and configure Go cache: -Follow these steps to securely pass your Go cache credentials into your Docker build: +```dockerfile +# syntax=docker/dockerfile:1 -1. Store the Depot token in a GitHub Secret named `DEPOT_TOKEN`. +# ... other Dockerfile instructions -2. Configure your GitHub Action to pass secrets to the container build: +# Install Depot CLI +RUN curl -L https://depot.dev/install-cli.sh | sh -```yaml -- name: Build and push - uses: depot/build-push-action@v1 - with: - context: . - file: ./Dockerfile - push: true - tags: your-image:tag - secrets: | - "DEPOT_TOKEN=${{ secrets.DEPOT_TOKEN }}" +# Mount secret and set GOCACHEPROG +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + PATH="/root/.depot/bin:$PATH" \ + GOCACHEPROG="depot gocache" \ + go build -v ./ ``` -3. Update your Dockerfile to install the Depot CLI and configure Go cache: +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +2. Build with Depot CLI: + +```shell +depot build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +Or with Docker Buildx: + +```shell +docker buildx build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +### Using Depot Cache with Go in Bake files + +When using Bake files to build Docker images containing Go projects, you can pass secrets through the `target.secret` attribute: + +1. Define the secret in your `docker-bake.hcl` file: + +```hcl +target "default" { + context = "." + dockerfile = "Dockerfile" + tags = ["your-image:tag"] + secret = [ + { + type = "env" + id = "DEPOT_TOKEN" + } + ] +} +``` + +2. Update your Dockerfile to install the Depot CLI and configure Go cache: ```dockerfile # syntax=docker/dockerfile:1 @@ -86,3 +160,9 @@ RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ ``` Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +3. Run the build with `depot bake`: + +```shell +DEPOT_TOKEN=your_token depot bake +``` diff --git a/content/cache/reference/gradle.mdx b/content/cache/reference/gradle.mdx index 591d092..91ea9c4 100644 --- a/content/cache/reference/gradle.mdx +++ b/content/cache/reference/gradle.mdx @@ -10,7 +10,7 @@ description: Learn how to use Depot remote caching for Gradle builds ## Configuring Gradle to use Depot Cache -Depot Cache can be used with Gradle from Depot's managed GitHub Actions runners, from your local machine, or from any CI/CD system. +Depot Cache can be used with Gradle from Depot's managed GitHub Actions runners, from your local machine, from any CI/CD system, or within containerized builds using Dockerfiles or Bake files. ### From Depot-managed Actions runners @@ -20,11 +20,78 @@ Depot Cache can be used with Gradle from Depot's managed GitHub Actions runners, org.gradle.caching=true ``` -If this automatic configuration is incompatible with your specific setup, you can disable automatic configuration in your organization settings page and manually configure Gradle to use Depot Cache as described below. +If you don't want Depot to override the `init.gradle` file on each runner, disable **Allow Actions jobs to automatically connect to Depot Cache** in your organization settings page. You can manually configure Gradle to use Depot Cache as described in the "Using Depot Cache from your local machine or any CI/CD system" section. -### From your local machine or any CI/CD system +### Using Depot Cache with Gradle in `depot/build-push-action` + +When using `depot/build-push-action` to build Docker images that contain Gradle projects, your build needs access to Gradle's remote cache credentials to benefit from caching. + +These credentials are not automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners (which have automatic access to Depot Cache environment variables), containerized builds execute in isolated VMs that require explicit configuration. + +Follow these steps to securely pass your Gradle credentials into your Docker build: + +1. Verify that caching is enabled in your `gradle.properties` file: -To manually configure Gradle to use Depot Cache, you will need to configure remote caching in your `settings.gradle` file. Configure Gradle to use the Depot Cache service endpoints and set your API token as the `password` credential: +```properties +org.gradle.caching=true +``` + +2. Store the Depot token in a GitHub Secret named `DEPOT_TOKEN`. + +3. Update your `settings.gradle` to read the Depot token from an environment variable: + +```groovy +buildCache { + remote(HttpBuildCache) { + url = 'https://cache.depot.dev' + enabled = true + push = true + credentials { + username = '' + password = System.getenv('DEPOT_TOKEN') + } + } +} +``` + +4. Configure your GitHub Action to pass secrets to the container build: + +```yaml +- name: Build and push + uses: depot/build-push-action@v1 + with: + context: . + file: ./Dockerfile + push: true + tags: your-image:tag + secrets: | + "DEPOT_TOKEN=${{ secrets.DEPOT_TOKEN }}" +``` + +5. Update your Dockerfile to mount the secret and run the build: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Copy Gradle configuration and run build with mounted secret +COPY gradle.properties settings.gradle ./ +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + ./gradlew build +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +### Using Depot Cache from your local machine or any CI/CD system + +To manually configure Gradle to use Depot Cache, you will need to configure remote caching in your `settings.gradle` file. First, verify that caching is enabled in your `gradle.properties` file: + +```properties +org.gradle.caching=true +``` + +Then, configure Gradle to use the Depot Cache service endpoints and set your API token as the `password` credential: `settings.gradle`: @@ -58,17 +125,17 @@ buildCache { } ``` -Once Gradle is configured to use Depot Cache, you can then run your builds as you normally would. Gradle will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. - -### Using Depot Cache with Gradle in `depot/build-push-action` +After Gradle is configured to use Depot Cache, you can then run your builds as you normally would. Gradle will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. -When using `depot/build-push-action` to build Docker images that contain Gradle projects, your build needs access to Gradle's remote cache credentials to benefit from caching. +### Using Depot Cache with Gradle in Depot CLI -These credentials are not automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners (which have automatic access to Depot Cache environment variables), containerized builds execute in isolated VMs that require explicit configuration. +When building directly with Depot CLI, follow these steps: -Follow these steps to securely pass your Gradle credentials into your Docker build: +1. Verify that caching is enabled in your `gradle.properties` file: -1. Store the Depot token in a GitHub Secret named `DEPOT_TOKEN`. +```properties +org.gradle.caching=true +``` 2. Update your `settings.gradle` to read the Depot token from an environment variable: @@ -86,31 +153,92 @@ buildCache { } ``` -3. Configure your GitHub Action to pass secrets to the container build: +3. Update your Dockerfile to copy Gradle configuration files and mount the secret: -```yaml -- name: Build and push - uses: depot/build-push-action@v1 - with: - context: . - file: ./Dockerfile - push: true - tags: your-image:tag - secrets: | - "DEPOT_TOKEN=${{ secrets.DEPOT_TOKEN }}" +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Copy Gradle configuration and run build with mounted secret +COPY gradle.properties settings.gradle ./ +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + ./gradlew build +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +4. Build with Depot CLI: + +```shell +depot build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +Or with Docker Buildx: + +```shell +docker buildx build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . ``` -4. Update your Dockerfile to mount the secret and run the build: +### Using Depot Cache with Gradle in Bake files + +When using Bake files to build Docker images containing Gradle projects, you can pass secrets through the `target.secret` attribute: + +1. Verify that caching is enabled in your `gradle.properties` file: + +```properties +org.gradle.caching=true +``` + +2. Update your `settings.gradle` to read the Depot token from an environment variable: + +```groovy +buildCache { + remote(HttpBuildCache) { + url = 'https://cache.depot.dev' + enabled = true + push = true + credentials { + username = '' + password = System.getenv('DEPOT_TOKEN') + } + } +} +``` + +3. Define the secret in your `docker-bake.hcl` file: + +```hcl +target "default" { + context = "." + dockerfile = "Dockerfile" + tags = ["your-image:tag"] + secret = [ + { + type = "env" + id = "DEPOT_TOKEN" + } + ] +} +``` + +4. Update your Dockerfile to copy Gradle configuration files and mount the secret: ```dockerfile # syntax=docker/dockerfile:1 # ... other Dockerfile instructions -# Copy settings.gradle and run build with mounted secret -COPY settings.gradle . +# Copy Gradle configuration and run build with mounted secret +COPY gradle.properties settings.gradle ./ RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ ./gradlew build ``` Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +5. Run the build with `depot bake`: + +```shell +DEPOT_TOKEN=your_token depot bake +``` diff --git a/content/cache/reference/maven.mdx b/content/cache/reference/maven.mdx index 9ed734e..2fdb369 100644 --- a/content/cache/reference/maven.mdx +++ b/content/cache/reference/maven.mdx @@ -10,7 +10,7 @@ description: Learn how to use Depot remote caching for Maven builds ## Configuring Maven to use Depot Cache -Depot Cache can be used with Maven from Depot's managed GitHub Actions runners, your local machine, or any CI/CD system. +Depot Cache can be used with Maven from Depot's managed GitHub Actions runners, your local machine, any CI/CD system, or within containerized builds using Dockerfiles or Bake files. ### From Depot-managed Actions runners @@ -48,9 +48,92 @@ You should also verify that you have registered the Build Cache extension in you ``` -If this automatic configuration is incompatible with your specific setup, you can disable automatic configuration in your organization settings page and manually configure Maven to use Depot Cache, as described below. +If you don't want Depot to override the Maven configuration files on each runner, disable **Allow Actions jobs to automatically connect to Depot Cache** in your organization settings page. You can manually configure Maven to use Depot Cache as described in the "Using Depot Cache from your local machine or any CI/CD system" section. -### From your local machine or any CI/CD system +### Using Depot Cache with Maven in `depot/build-push-action` + +When using `depot/build-push-action` to build Docker images that contain Maven projects, your build needs access to Maven's remote cache credentials to benefit from caching. + +These credentials are not automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners (which have automatic access to Depot Cache environment variables), containerized builds execute in isolated VMs that require explicit configuration. + +Follow these steps to securely pass your Maven credentials into your Docker build: + +1. Store the Depot token in a GitHub Secret named `DEPOT_TOKEN`. + +2. Configure Maven Build Cache extension in `.mvn/maven-build-cache-config.xml`: + +```xml + + + true + SHA-256 + true + + https://cache.depot.dev + + + + +``` + +3. Update your `settings.xml` to read the Depot token from an environment variable. Create or update `.m2/settings.xml`: + +```xml + + + + + depot-cache + + + + Authorization + Bearer ${env.DEPOT_TOKEN} + + + + + + +``` + +4. Configure your GitHub Action to pass secrets to the container build: + +```yaml +- name: Build and push + uses: depot/build-push-action@v1 + with: + context: . + file: ./Dockerfile + push: true + tags: your-image:tag + secrets: | + "DEPOT_TOKEN=${{ secrets.DEPOT_TOKEN }}" +``` + +5. Update your Dockerfile to copy configuration files and run the build with mounted secret: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Copy Maven configuration files +COPY .mvn/maven-build-cache-config.xml .mvn/maven-build-cache-config.xml +COPY .m2/settings.xml /root/.m2/settings.xml + +# Run build with mounted secret +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + mvn clean install +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +### Using Depot Cache from your local machine or any CI/CD system To manually configure Maven to use Depot Cache, you will need to configure remote caching in your `~/.m2/settings.xml` file. Configure Maven to use the Depot Cache service endpoints and set your API token where there is the `DEPOT_TOKEN` below: @@ -77,19 +160,88 @@ To manually configure Maven to use Depot Cache, you will need to configure remot **Note: Maven support currently only supports Depot Organization API tokens, not user tokens.** -Once Maven is configured to use Depot Cache, you can run your builds as usual. Maven will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. +After Maven is configured to use Depot Cache, you can run your builds as usual. Maven will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. -### Using Depot Cache with Maven in `depot/build-push-action` +### Using Depot Cache with Maven in Depot CLI -When using `depot/build-push-action` to build Docker images that contain Maven projects, your build needs access to Maven's remote cache credentials to benefit from caching. +When building directly with Depot CLI, follow these steps: -These credentials are not automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners (which have automatic access to Depot Cache environment variables), containerized builds execute in isolated VMs that require explicit configuration. +1. Configure Maven Build Cache extension in `.mvn/maven-build-cache-config.xml`: -Follow these steps to securely pass your Maven credentials into your Docker build: +```xml + + + true + SHA-256 + true + + https://cache.depot.dev + + + + +``` -1. Store the Depot token in a GitHub Secret named `DEPOT_TOKEN`. +2. Update your `settings.xml` to read the Depot token from an environment variable. Create or update `.m2/settings.xml`: -2. Configure Maven Build Cache extension in `.mvn/maven-build-cache-config.xml`: +```xml + + + + + depot-cache + + + + Authorization + Bearer ${env.DEPOT_TOKEN} + + + + + + +``` + +3. Update your Dockerfile to copy configuration files and mount the secret: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Copy Maven configuration files +COPY .mvn/maven-build-cache-config.xml .mvn/maven-build-cache-config.xml +COPY .m2/settings.xml /root/.m2/settings.xml + +# Run build with mounted secret +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + mvn clean install +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +4. Build with Depot CLI: + +```shell +depot build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +Or with Docker Buildx: + +```shell +docker buildx build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +### Using Depot Cache with Maven in Bake files + +When using Bake files to build Docker images containing Maven projects, you can pass secrets through the `target.secret` attribute: + +1. Configure Maven Build Cache extension in `.mvn/maven-build-cache-config.xml`: ```xml ``` -3. Update your `settings.xml` to read the Depot token from an environment variable. Create or update `.m2/settings.xml`: +2. Update your `settings.xml` to read the Depot token from an environment variable. Create or update `.m2/settings.xml`: ```xml @@ -130,21 +282,23 @@ Follow these steps to securely pass your Maven credentials into your Docker buil ``` -4. Configure your GitHub Action to pass secrets to the container build: - -```yaml -- name: Build and push - uses: depot/build-push-action@v1 - with: - context: . - file: ./Dockerfile - push: true - tags: your-image:tag - secrets: | - "DEPOT_TOKEN=${{ secrets.DEPOT_TOKEN }}" +3. Define the secret in your `docker-bake.hcl` file: + +```hcl +target "default" { + context = "." + dockerfile = "Dockerfile" + tags = ["your-image:tag"] + secret = [ + { + type = "env" + id = "DEPOT_TOKEN" + } + ] +} ``` -5. Update your Dockerfile to copy configuration files and run the build with mounted secret: +4. Update your Dockerfile to copy configuration files and run the build with mounted secret: ```dockerfile # syntax=docker/dockerfile:1 @@ -161,3 +315,9 @@ RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ ``` Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +5. Run the build with `depot bake`: + +```shell +DEPOT_TOKEN=your_token depot bake +``` diff --git a/content/cache/reference/moonrepo.mdx b/content/cache/reference/moonrepo.mdx index e13f150..f2bab5e 100644 --- a/content/cache/reference/moonrepo.mdx +++ b/content/cache/reference/moonrepo.mdx @@ -10,31 +10,13 @@ description: Learn how to use Depot remote caching for moonrepo builds ## Configuring moonrepo to use Depot Cache -Depot Cache can be used with moonrepo from Depot's managed GitHub Actions runners, from your local machine, or from any CI/CD system. +Depot Cache can be used with moonrepo from Depot's managed GitHub Actions runners, from your local machine, from any CI/CD system, or within containerized builds using Dockerfiles or Bake files. -To configure `moon` to use Depot Cache, you will need to set a `DEPOT_TOKEN` environment variable with an organization or user token and add the following to your `.moon/workspace.yml` file: +### From Depot-managed Actions runners -```yaml -unstable_remote: - host: 'grpcs://cache.depot.dev' - auth: - token: 'DEPOT_TOKEN' -``` +[Depot GitHub Actions runners](/docs/github-actions/overview) are pre-configured to use Depot Cache with moonrepo - each runner is launched with the necessary environment variables for accessing Depot Cache. -If you are using a user token and are a member of more than one organization, you will additionally need to set an `X-Depot-Org` header to your Depot organization ID in `.moon/workspace.yml`: - -```yaml -unstable_remote: - host: 'grpcs://cache.depot.dev' - auth: - token: 'DEPOT_TOKEN' - headers: - 'X-Depot-Org': '' -``` - -See [moonrepo's remote cache documentation](https://moonrepo.dev/docs/guides/remote-cache#cloud-hosted-depot) for more details. - -Once moonrepo is configured to use Depot Cache, you can then run your builds as you normally would. moonrepo will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. +If you don't want Depot to override the moonrepo workspace configuration on each runner, disable **Allow Actions jobs to automatically connect to Depot Cache** in your organization settings page. You can manually configure moonrepo to use Depot Cache as described in the "Using Depot Cache from your local machine or any CI/CD system" section. ### Using Depot Cache with moonrepo in `depot/build-push-action` @@ -85,3 +67,123 @@ RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ ``` Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +### Using Depot Cache from your local machine or any CI/CD system + +To manually configure `moon` to use Depot Cache, you will need to set a `DEPOT_TOKEN` environment variable with an organization or user token and add the following to your `.moon/workspace.yml` file: + +```yaml +unstable_remote: + host: 'grpcs://cache.depot.dev' + auth: + token: 'DEPOT_TOKEN' +``` + +If you are using a user token and are a member of more than one organization, you will additionally need to set an `X-Depot-Org` header to your Depot organization ID in `.moon/workspace.yml`: + +```yaml +unstable_remote: + host: 'grpcs://cache.depot.dev' + auth: + token: 'DEPOT_TOKEN' + headers: + 'X-Depot-Org': '' +``` + +See [moonrepo's remote cache documentation](https://moonrepo.dev/docs/guides/remote-cache#cloud-hosted-depot) for more details. + +After moonrepo is configured to use Depot Cache, you can then run your builds as you normally would. moonrepo will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. + +### Using Depot Cache with moonrepo in Depot CLI + +When building directly with Depot CLI, follow these steps: + +1. Configure moonrepo to read the Depot token from an environment variable in `.moon/workspace.yml`: + +```yaml +unstable_remote: + host: 'grpcs://cache.depot.dev' + auth: + token: 'DEPOT_TOKEN' +``` + +2. Update your Dockerfile to copy the workspace configuration and mount the secret: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Copy moonrepo workspace configuration +COPY .moon/workspace.yml .moon/workspace.yml + +# Mount secret as environment variable and run build +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + moon run build +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +3. Build with Depot CLI: + +```shell +depot build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +Or with Docker Buildx: + +```shell +docker buildx build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +### Using Depot Cache with moonrepo in Bake files + +When using Bake files to build Docker images containing moonrepo workspaces, you can pass secrets through the `target.secret` attribute: + +1. Configure moonrepo to read the Depot token from an environment variable in `.moon/workspace.yml`: + +```yaml +unstable_remote: + host: 'grpcs://cache.depot.dev' + auth: + token: 'DEPOT_TOKEN' +``` + +2. Define the secret in your `docker-bake.hcl` file: + +```hcl +target "default" { + context = "." + dockerfile = "Dockerfile" + tags = ["your-image:tag"] + secret = [ + { + type = "env" + id = "DEPOT_TOKEN" + } + ] +} +``` + +3. Update your Dockerfile to copy the workspace configuration and run the build with mounted secret: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Copy moonrepo workspace configuration +COPY .moon/workspace.yml .moon/workspace.yml + +# Mount secret as environment variable and run build +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + moon run build +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +4. Run the build with `depot bake`: + +```shell +DEPOT_TOKEN=your_token depot bake +``` diff --git a/content/cache/reference/pants.mdx b/content/cache/reference/pants.mdx index 755a0c7..8efc918 100644 --- a/content/cache/reference/pants.mdx +++ b/content/cache/reference/pants.mdx @@ -10,15 +10,66 @@ description: Learn how to use Depot remote caching for Pants builds ## Configuring Pants to use Depot Cache -Depot Cache can be used with Pants from Depot's managed GitHub Actions runners, from your local machine, or from any CI/CD system. +Depot Cache can be used with Pants from Depot's managed GitHub Actions runners, from your local machine, from any CI/CD system, or within containerized builds using Dockerfiles or Bake files. ### From Depot-managed Actions runners [Depot GitHub Actions runners](/docs/github-actions/overview) are pre-configured to use Depot Cache with Pants - each runner is launched with a `pants.toml` file that is pre-configured with the connection details for Depot Cache. -If this automatic configuration is incompatible with your specific setup, you can disable automatic configuration in your organization settings page and manually configure Pants to use Depot Cache as described below. +If you don't want Depot to override the `pants.toml` file on each runner, disable **Allow Actions jobs to automatically connect to Depot Cache** in your organization settings page. You can manually configure Pants to use Depot Cache as described in the "Using Depot Cache from your local machine or any CI/CD system" section. -### From your local machine or any CI/CD system +### Using Depot Cache with Pants in `depot/build-push-action` + +When using `depot/build-push-action` to build Docker images that contain Pants projects, your build needs access to Pants' remote cache credentials to benefit from caching. + +These credentials are not automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners (which have automatic access to Depot Cache environment variables), containerized builds execute in isolated VMs that require explicit configuration. + +Follow these steps to securely pass your Pants credentials into your Docker build: + +1. Store the Depot token in a GitHub Secret named `DEPOT_TOKEN`. + +2. Update your `pants.toml` to read the Depot token from an environment variable: + +```toml +[GLOBAL] +remote_cache_read = true +remote_cache_write = true +remote_store_address = "grpcs://cache.depot.dev" + +[GLOBAL.remote_store_headers] +Authorization = "%(env.DEPOT_TOKEN)s" +``` + +3. Configure your GitHub Action to pass secrets to the container build: + +```yaml +- name: Build and push + uses: depot/build-push-action@v1 + with: + context: . + file: ./Dockerfile + push: true + tags: your-image:tag + secrets: | + "DEPOT_TOKEN=${{ secrets.DEPOT_TOKEN }}" +``` + +4. Update your Dockerfile to mount the secret and run the build: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Copy pants.toml and run build with mounted secret +COPY pants.toml . +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + pants package :: +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +### Using Depot Cache from your local machine or any CI/CD system To manually configure Pants to use Depot Cache, you will need to enable remote caching in your `pants.toml`. Configure Pants to use the Depot Cache service endpoints and set your API token in the `Authorization` header: @@ -41,19 +92,56 @@ If you are a member of multiple organizations, and you are authenticating with a remote_store_headers = { "x-depot-org" = "DEPOT_ORG_ID" } ``` -Once Pants is configured to use Depot Cache, you can then run your builds as you normally would. Pants will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. +After Pants is configured to use Depot Cache, you can then run your builds as you normally would. Pants will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. -### Using Depot Cache with Pants in `depot/build-push-action` +### Using Depot Cache with Pants in Depot CLI -When using `depot/build-push-action` to build Docker images that contain Pants projects, your build needs access to Pants' remote cache credentials to benefit from caching. +When building directly with Depot CLI, follow these steps: -These credentials are not automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners (which have automatic access to Depot Cache environment variables), containerized builds execute in isolated VMs that require explicit configuration. +1. Update your `pants.toml` to read the Depot token from an environment variable: -Follow these steps to securely pass your Pants credentials into your Docker build: +```toml +[GLOBAL] +remote_cache_read = true +remote_cache_write = true +remote_store_address = "grpcs://cache.depot.dev" -1. Store the Depot token in a GitHub Secret named `DEPOT_TOKEN`. +[GLOBAL.remote_store_headers] +Authorization = "%(env.DEPOT_TOKEN)s" +``` -2. Update your `pants.toml` to read the Depot token from an environment variable: +2. Update your Dockerfile to copy the configuration and mount the secret: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Copy pants.toml and run build with mounted secret +COPY pants.toml . +RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ + pants package :: +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +3. Build with Depot CLI: + +```shell +depot build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +Or with Docker Buildx: + +```shell +docker buildx build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +### Using Depot Cache with Pants in Bake files + +When using Bake files to build Docker images containing Pants projects, you can pass secrets through the `target.secret` attribute: + +1. Update your `pants.toml` to read the Depot token from an environment variable: ```toml [GLOBAL] @@ -65,21 +153,23 @@ remote_store_address = "grpcs://cache.depot.dev" Authorization = "%(env.DEPOT_TOKEN)s" ``` -3. Configure your GitHub Action to pass secrets to the container build: - -```yaml -- name: Build and push - uses: depot/build-push-action@v1 - with: - context: . - file: ./Dockerfile - push: true - tags: your-image:tag - secrets: | - "DEPOT_TOKEN=${{ secrets.DEPOT_TOKEN }}" +2. Define the secret in your `docker-bake.hcl` file: + +```hcl +target "default" { + context = "." + dockerfile = "Dockerfile" + tags = ["your-image:tag"] + secret = [ + { + type = "env" + id = "DEPOT_TOKEN" + } + ] +} ``` -4. Update your Dockerfile to mount the secret and run the build: +3. Update your Dockerfile to copy the configuration and mount the secret: ```dockerfile # syntax=docker/dockerfile:1 @@ -93,3 +183,9 @@ RUN --mount=type=secret,id=DEPOT_TOKEN,env=DEPOT_TOKEN \ ``` Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +4. Run the build with `depot bake`: + +```shell +DEPOT_TOKEN=your_token depot bake +``` diff --git a/content/cache/reference/sccache.mdx b/content/cache/reference/sccache.mdx index e9fd0ea..f4e40bb 100644 --- a/content/cache/reference/sccache.mdx +++ b/content/cache/reference/sccache.mdx @@ -10,32 +10,13 @@ description: Learn how to use Depot remote caching for sccache builds ## Configuring sccache to use Depot Cache -Depot Cache can be used with sccache from Depot's managed GitHub Actions runners, from your local machine, or from any CI/CD system. +Depot Cache can be used with sccache from Depot's managed GitHub Actions runners, from your local machine, from any CI/CD system, or within containerized builds using Dockerfiles or Bake files. ### From Depot-managed Actions runners [Depot GitHub Actions runners](/docs/github-actions/overview) are pre-configured to use Depot Cache with sccache - each runner is launched with a `SCCACHE_WEBDAV_ENDPOINT` environment variable and is pre-configured with the connection details for Depot Cache. -If this automatic configuration is incompatible with your specific setup, you can disable automatic configuration in your organization settings page and manually configure sccache to use Depot Cache as described below. - -### From your local machine or any CI/CD system - -To manually configure sccache to use Depot Cache, you will need to set two environment variables in your environment, representing the Depot Cache service endpoint and your API token: - -```shell -export SCCACHE_WEBDAV_ENDPOINT=https://cache.depot.dev -export SCCACHE_WEBDAV_TOKEN=DEPOT_TOKEN -``` - -If you are a member of multiple organizations, and you are authenticating with a user token, you must instead specify a password along with which organization should be used for cache storage as follows: - -```shell -export SCCACHE_WEBDAV_ENDPOINT=https://cache.depot.dev -export SCCACHE_WEBDAV_USERNAME=DEPOT_ORG_ID -export SCCACHE_WEBDAV_PASSWORD=DEPOT_TOKEN -``` - -Once sccache is configured to use Depot Cache, you can then run your builds as you normally would. sccache will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. +If you don't want Depot to set up the `SCCACHE_WEBDAV_ENDPOINT` environment variable on each runner, disable **Allow Actions jobs to automatically connect to Depot Cache** in your organization settings page. You can manually configure sccache to use Depot Cache as described in the "Using Depot Cache from your local machine or any CI/CD system" section. ### Using Depot Cache with sccache in `depot/build-push-action` @@ -75,3 +56,94 @@ RUN --mount=type=secret,id=DEPOT_TOKEN,env=SCCACHE_WEBDAV_TOKEN \ ``` Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +### Using Depot Cache from your local machine or any CI/CD system + +To manually configure sccache to use Depot Cache, you will need to set two environment variables in your environment, representing the Depot Cache service endpoint and your API token: + +```shell +export SCCACHE_WEBDAV_ENDPOINT=https://cache.depot.dev +export SCCACHE_WEBDAV_TOKEN=DEPOT_TOKEN +``` + +If you are a member of multiple organizations, and you are authenticating with a user token, you must instead specify a password along with which organization should be used for cache storage as follows: + +```shell +export SCCACHE_WEBDAV_ENDPOINT=https://cache.depot.dev +export SCCACHE_WEBDAV_USERNAME=DEPOT_ORG_ID +export SCCACHE_WEBDAV_PASSWORD=DEPOT_TOKEN +``` + +After sccache is configured to use Depot Cache, you can then run your builds as you normally would. sccache will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. + +### Using Depot Cache with sccache in Depot CLI + +When building directly with Depot CLI, follow these steps: + +1. Update your Dockerfile to mount the secret as environment variables: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Mount secrets with IDs matching the environment variable names +RUN --mount=type=secret,id=DEPOT_TOKEN,env=SCCACHE_WEBDAV_TOKEN \ + SCCACHE_WEBDAV_ENDPOINT=https://cache.depot.dev sccache --start-server && \ + cargo build --release +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +2. Build with Depot CLI: + +```shell +depot build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +Or with Docker Buildx: + +```shell +docker buildx build --secret id=DEPOT_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +### Using Depot Cache with sccache in Bake files + +When using Bake files to build Docker images containing Rust projects with sccache, you can pass secrets through the `target.secret` attribute: + +1. Define the secret in your `docker-bake.hcl` file: + +```hcl +target "default" { + context = "." + dockerfile = "Dockerfile" + tags = ["your-image:tag"] + secret = [ + { + type = "env" + id = "DEPOT_TOKEN" + } + ] +} +``` + +2. Update your Dockerfile to mount the secret as environment variables: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Mount secrets with IDs matching the environment variable names +RUN --mount=type=secret,id=DEPOT_TOKEN,env=SCCACHE_WEBDAV_TOKEN \ + SCCACHE_WEBDAV_ENDPOINT=https://cache.depot.dev sccache --start-server && \ + cargo build --release +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +3. Run the build with `depot bake`: + +```shell +DEPOT_TOKEN=your_token depot bake +``` diff --git a/content/cache/reference/turbo.mdx b/content/cache/reference/turbo.mdx index 4256621..ab793c8 100644 --- a/content/cache/reference/turbo.mdx +++ b/content/cache/reference/turbo.mdx @@ -4,46 +4,32 @@ ogTitle: Remote caching for Turborepo builds description: Learn how to use Depot remote caching for Turborepo builds --- -[**Turborepo**](https://turbo.build/) is a high-performance build system for JavaScript and TypeScript codebases, and is designed around scaling build performance for large monorepos. It is used by large projects at Netflix, AWS, and Disney, and supports incremental builds backed by local and remote cache options. +[**Turborepo**](https://turbo.build/) is a high-performance build system for JavaScript and TypeScript codebases, designed to scale build performance for large monorepos. Large projects at Netflix, AWS, and Disney use it, and it supports incremental builds backed by local and remote cache options. -[**Depot Cache**](/docs/cache/overview) provides a remote cache service that can be used with Turborepo, allowing you to incrementally cache and reuse parts of your builds. This cache is accessible from anywhere, both on your local machine and on CI/CD systems. +[**Depot Cache**](/docs/cache/overview) provides a remote cache service that works with Turborepo, allowing you to incrementally cache and reuse parts of your builds. This cache is accessible from anywhere, both on your local machine and on CI/CD systems. -## Configuring Turborepo to use Depot Cache +## Configure Turborepo to use Depot Cache -Depot Cache can be used with Turborepo from Depot's managed GitHub Actions runners, from your local machine, or from any CI/CD system. +Use Depot Cache with Turborepo from Depot's managed GitHub Actions runners, from your local machine, from any CI/CD system, or within containerized builds using Dockerfiles or Bake files. ### From Depot-managed Actions runners -[Depot GitHub Actions runners](/docs/github-actions/overview) are pre-configured to use Depot Cache with Turborepo - each runner is launched with a `TURBO_API` environment variable and is pre-configured with the connection details for Depot Cache. +[Depot GitHub Actions runners](/docs/github-actions/overview) are pre-configured to use Depot Cache with Turborepo - each runner launches with a `TURBO_API` environment variable and includes the connection details for Depot Cache. -If this automatic configuration is incompatible with your specific setup, you can disable automatic configuration in your organization settings page and manually configure Turborepo to use Depot Cache as described below. +If you don't want Depot to set up the Turborepo environment variables on each runner, turn off **Allow Actions jobs to automatically connect to Depot Cache** in your organization settings page. You can manually configure Turborepo to use Depot Cache as described in the "Use Depot Cache from your local machine or any CI/CD system" section. -### From your local machine or any CI/CD system - -To manually configure Turborepo to use Depot Cache, you will need to set three environment variables in your environment. These represent the Depot Cache service endpoint, your API token, and your Depot organization id: - -```shell -export TURBO_API=https://cache.depot.dev -export TURBO_TOKEN=DEPOT_TOKEN -export TURBO_TEAM=DEPOT_ORG_ID -``` - -Once Turborepo is configured to use Depot Cache, you can then run your builds as you normally would. Turborepo will automatically communicate with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. - -### Using Depot Cache with Turborepo in `depot/build-push-action` +### Use Depot Cache with Turborepo in `depot/build-push-action` When using `depot/build-push-action` to build Docker images that contain Turborepo workspaces, your build needs access to Turborepo's remote cache credentials to benefit from caching. -These credentials are not automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners (which have automatic access to Depot Cache environment variables), containerized builds execute in isolated VMs that require explicit configuration. +These credentials aren't automatically available inside your Docker build environment. Unlike builds running directly on Depot-managed GitHub Actions runners, which have automatic access to Depot Cache environment variables, containerized builds execute in isolated VMs that require explicit configuration. -Follow these steps to securely pass your Turborepo credentials into your Docker build: +Follow these steps to pass your Turborepo credentials into your Docker build: -1. Create GitHub Secrets for your Turborepo cache variables: - - `TURBO_API` - - `TURBO_TOKEN` - - `TURBO_TEAM` +1. Create a GitHub Secret for your Depot token: + - `DEPOT_TOKEN` -2. Configure your GitHub Action to pass secrets to the container build: +2. Configure your GitHub Action to pass the token as a secret: ```yaml - name: Build and push @@ -54,23 +40,116 @@ Follow these steps to securely pass your Turborepo credentials into your Docker push: true tags: your-image:tag secrets: | - "TURBO_API=${{ secrets.TURBO_API }}" - "TURBO_TOKEN=${{ secrets.TURBO_TOKEN }}" - "TURBO_TEAM=${{ secrets.TURBO_TEAM }}" + "TURBO_TOKEN=${{ secrets.DEPOT_TOKEN }}" +``` + +3. Update your Dockerfile to configure Depot Cache and mount the token as a secret: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Configure Depot Cache for Turborepo +ENV TURBO_API=https://cache.depot.dev +ENV TURBO_TEAM=your_org_id + +# Mount the token secret +RUN --mount=type=secret,id=TURBO_TOKEN,env=TURBO_TOKEN \ + turbo build +``` + +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +### Use Depot Cache from your local machine or any CI/CD system + +To manually configure Turborepo to use Depot Cache, set three environment variables in your environment. These represent the Depot Cache service endpoint, your API token, and your Depot organization id: + +```shell +export TURBO_API=https://cache.depot.dev +export TURBO_TOKEN=DEPOT_TOKEN +export TURBO_TEAM=DEPOT_ORG_ID +``` + +After you configure Turborepo to use Depot Cache, run your builds as you normally would. Turborepo automatically communicates with Depot Cache to fetch and reuse any stored build artifacts from your previous builds. + +### Use Depot Cache with Turborepo in Depot CLI + +When building directly with Depot CLI, follow these steps: + +1. Update your Dockerfile to configure Depot Cache and mount the token as a secret: + +```dockerfile +# syntax=docker/dockerfile:1 + +# ... other Dockerfile instructions + +# Configure Depot Cache for Turborepo +ENV TURBO_API=https://cache.depot.dev +ENV TURBO_TEAM=your_org_id + +# Mount the token secret +RUN --mount=type=secret,id=TURBO_TOKEN,env=TURBO_TOKEN \ + turbo build ``` -3. Update your Dockerfile to mount the secrets as environment variables: +Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +2. Build with Depot CLI: + +```shell +export DEPOT_TOKEN=your_token +depot build --secret id=TURBO_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +Or with Docker Buildx: + +```shell +export DEPOT_TOKEN=your_token +docker buildx build --secret id=TURBO_TOKEN,env=DEPOT_TOKEN -t your-image:tag . +``` + +### Use Depot Cache with Turborepo in Bake files + +When using Bake files to build Docker images containing Turborepo workspaces, you can pass the token as a secret: + +1. Define the configuration in your `docker-bake.hcl` file: + +```hcl +target "default" { + context = "." + dockerfile = "Dockerfile" + tags = ["your-image:tag"] + secret = [ + { + type = "env" + id = "DEPOT_TOKEN" + } + ] +} +``` + +2. Update your Dockerfile to configure Depot Cache and mount the token as a secret: ```dockerfile # syntax=docker/dockerfile:1 # ... other Dockerfile instructions -# Mount secrets with IDs matching the environment variable names -RUN --mount=type=secret,id=TURBO_API,env=TURBO_API \ - --mount=type=secret,id=TURBO_TOKEN,env=TURBO_TOKEN \ - --mount=type=secret,id=TURBO_TEAM,env=TURBO_TEAM \ +# Configure Depot Cache for Turborepo +ENV TURBO_API=https://cache.depot.dev +ENV TURBO_TEAM=your_org_id + +# Mount the token secret +RUN --mount=type=secret,id=DEPOT_TOKEN,env=TURBO_TOKEN \ turbo build ``` Adding `# syntax=docker/dockerfile:1` as the first line of your Dockerfile enables mounting secrets as environment variables. + +3. Run the build with `depot bake`: + +```shell +export DEPOT_TOKEN=your_token +depot bake +``` diff --git a/content/container-builds/how-to-guides/autoscaling.mdx b/content/container-builds/how-to-guides/autoscaling.mdx index c10fade..0b1c370 100644 --- a/content/container-builds/how-to-guides/autoscaling.mdx +++ b/content/container-builds/how-to-guides/autoscaling.mdx @@ -140,6 +140,16 @@ Here's an example of when autoscaling might be beneficial: Result: Each build runs with dedicated resources, preventing resource contention and ensuring optimal performance even during peak usage. +## Understanding `depot bake` and autoscaling + +A `depot bake` command is submitted as a single build request to BuildKit, regardless of how many targets are defined in the bake file. This means: + +- For autoscaling purposes, one `depot bake` command counts as one build, not multiple builds +- For example, if your project has Autoscaling enabled with a value of `2` builds per instance, two concurrent `depot bake` commands will run on the same builder, but a third concurrent `depot bake` command will trigger the provisioning of a new builder +- The number of targets inside a bake file doesn't affect the autoscaling count + +**Splitting bake builds across projects**: You can specify different project IDs to split a single bake into multiple builds (one per project). However, the number of targets inside the bake for each project has no impact on autoscaling. Each `depot bake` command for each project still counts as a single build. + ## Troubleshooting If you're experiencing issues with autoscaling: diff --git a/content/container-builds/how-to-guides/docker-bake.mdx b/content/container-builds/how-to-guides/docker-bake.mdx index 39c5bdd..ebead73 100644 --- a/content/container-builds/how-to-guides/docker-bake.mdx +++ b/content/container-builds/how-to-guides/docker-bake.mdx @@ -247,6 +247,18 @@ target "worker" { } ``` +**Note:** When you use the `depot/bake-action` in a GitHub Actions workflow, the `x-depot.project-id` in your Docker Compose file or `project_id` in your HCL bake file take precedence over the `project` input in the action configuration. + +### Understanding bake and autoscaling + +A `depot bake` command is submitted as a single build request to BuildKit, regardless of how many targets are defined in the bake file. This has important implications for [build autoscaling](/docs/container-builds/how-to-guides/autoscaling): + +- For autoscaling purposes, one `depot bake` command counts as one build, not multiple builds +- For example, if your project has Autoscaling enabled with a value of `2` builds per instance, two concurrent `depot bake` commands will run on the same builder, but a third concurrent `depot bake` command will trigger the provisioning of a new builder +- The number of targets inside a bake file has no impact on the autoscaling count + +**Splitting bake builds across projects**: You can specify different project IDs (as shown in the section above) to split a single bake into multiple builds (one per project). However, the number of targets inside the bake for each project has no impact on autoscaling. Each `depot bake` command for each project still counts as a single build. + ### Loading images locally Load specific targets to your local Docker daemon by including the target name after the load flag: diff --git a/content/container-builds/how-to-guides/docker-compose.mdx b/content/container-builds/how-to-guides/docker-compose.mdx index ffe1ab1..d6e943b 100644 --- a/content/container-builds/how-to-guides/docker-compose.mdx +++ b/content/container-builds/how-to-guides/docker-compose.mdx @@ -66,6 +66,8 @@ services: With the above configuration, the `app` service will be built in the `abc123456` Depot project and the `backend` service will be built in the `xyz123456` Depot project when running `depot bake`. +**Note:** When you use the `depot/bake-action` in a GitHub Actions workflow, the `x-depot.project-id` in your Docker Compose file takes precedence over the `project` input in the action configuration. + ## Building images with `docker compose build` If you are unable to use `depot bake --load` and need to use `docker compose build` directly, you can still use Depot to accelerate your builds. Docker Compose can use Docker Buildx to build the requested images in the `docker-compose.yml` file, and Depot can be installed as a Buildx driver serve those build requests. @@ -111,7 +113,7 @@ permissions: jobs: build-services: - runs-on: ubuntu-22.04 + runs-on: ubuntu-24.04 outputs: build-id: ${{ steps.bake.outputs.build-id }} steps: @@ -125,7 +127,7 @@ jobs: save: true test: - runs-on: depot-ubuntu-22.04 + runs-on: depot-ubuntu-24.04 needs: [build-services] steps: - uses: actions/checkout@v4 diff --git a/content/container-builds/how-to-guides/private-registries.mdx b/content/container-builds/how-to-guides/private-registries.mdx index 4eeec42..8181b80 100644 --- a/content/container-builds/how-to-guides/private-registries.mdx +++ b/content/container-builds/how-to-guides/private-registries.mdx @@ -1,16 +1,16 @@ --- -title: Accessing Private Registries +title: Access private registries ogTitle: How to build images that can access private registries with Depot -description: Learn how to build images with Depot that need to access private registry images within them. +description: Learn how to build container images with Depot that can access private registries. --- -## How do I build container images that access private registries? - -Our `depot` CLI uses your local Docker credentials provider. So, any registry you've logged into with `docker login` or similar will be available when running a Depot build. This means that you can build images that use private registries like the example below. +The Depot CLI uses your local Docker credentials provider. To allow your Depot build to access a private image registry, log into the registry with `docker login registry.example.com` on the same machine where you'll run `depot build`. Use the private registry in your Dockerfile as you normally would: ```dockerfile FROM my-private-registry/project/image:version -... +# rest of your Dockerfile ``` -If you are experiencing issues with this, you should confirm you have logged into the registry from the machine where you are trying to run `depot build`. One way to ensure this is to try pulling the image via `docker pull my-private-registry/project/image:version`. +Then run `depot build .`. + +You should be able to use any registry with no additional Depot configuration. If you're having issues accessing private registries from your builds, see [Cannot pull from private registry during build](/docs/container-builds/troubleshooting#cannot-pull-from-private-registry-during-build). diff --git a/content/container-builds/troubleshooting.mdx b/content/container-builds/troubleshooting.mdx index 7555bd7..aeb9b9d 100644 --- a/content/container-builds/troubleshooting.mdx +++ b/content/container-builds/troubleshooting.mdx @@ -1,33 +1,244 @@ --- -title: Troubleshooting -ogTitle: How to troubleshoot common problems using Depot -description: Learn about common errors with Depot and how to resolve them. +title: Troubleshooting container builds +ogTitle: Troubleshooting Depot container builds +description: Common errors and how to resolve them when building container images with Depot. --- -## Resetting build cache for a project +This page provides an overview of common errors encountered when building container images with Depot, along with steps to resolve them. -Each project in Depot represents a cache namespace shared among all systems and users that build it. The cache allows certain steps in your `Dockerfile` to be skipped in subsequent runs if their associated layer hasn't changed since the last time it was built. There can be times when you want to purge this cache. +## Error: `Keep alive ping failed to receive ACK within timeout` -- The build is hung because of a deadlock in BuildKit -- A builder isn't coming online to serve the build request -- The build cache is full and needs to be cleared +This error occurs when BuildKit is shut down due to runner resource starvation, often caused by an Out of Memory (OOM) condition. -We optimize Depot so that these issues rarely happen. However, if you see any of these issues, you can reset the build cache for a project. Resetting the build cache purges the cache volume and launches a new build machine with a clean slate. You can reset the build cache for a project via these steps: +### How to resolve + +To resolve this issue, try one of the following configuration changes: + +- **Scale up your worker size:** Increase the resources available to each build by selecting a larger worker size in your project settings. + +- **Enable auto-scaling:** Limit the number of builds running simultaneously on a given worker to prevent resource contention. For more information about auto-scaling, see the [Auto-scaling Guide](https://depot.dev/docs/container-builds/how-to-guides/autoscaling). + +If you continue to experience this error after adjusting your worker configuration, [reach out to support](/help) with your project ID and build details so we can help investigate resource usage patterns. + +## Error: `Unable to acquire machine, please retry` + +If you encounter a container build failure with the error message `Unable to acquire machine, please retry`, this indicates an issue with machine availability or BuildKit responsiveness. This error can occur for two main reasons: + +**1. Regional capacity or maintenance issues:** + +- Scheduled maintenance events in a specific region +- Temporary capacity constraints +- Isolated incidents affecting specific projects + +**2. BuildKit resource exhaustion:** + +- BuildKit stops responding due to being overwhelmed with too many concurrent builds +- New builds wait for BuildKit's health checks before starting +- If BuildKit doesn't report healthy within 5 minutes, builds fail with this error + +This is often the same underlying cause as the `Keep alive ping failed to receive ACK within timeout` error. When BuildKit is overwhelmed, existing builds may see keep-alive ping failures, while new builds attempting to connect see this `unable to acquire machine` error. + +### How to resolve + +First, check [status.depot.dev](https://status.depot.dev) for any reported outages. If there's an active incident in a certain region, you can switch your project to a different region temporarily. + +If there are no reported incidents, this is likely a resource exhaustion issue. Try one of these configuration changes: + +- **Scale up your worker size:** Increase the resources available to each build by selecting a larger worker size in your project settings. + +- **Enable auto-scaling:** Limit the number of builds running simultaneously on a given worker to prevent resource contention. For more information about auto-scaling, see the [Auto-scaling Guide](https://depot.dev/docs/container-builds/how-to-guides/autoscaling). + +If you continue experiencing this error after checking for incidents and adjusting your worker configuration, [reach out to support](/help) with: + +- Your project ID +- Whether the error occurs consistently or intermittently +- How many concurrent builds typically run on your project + +## Error: `Our services aren't available right now` + +When building images with Depot, you may see an error message similar to: + +```text +Error: failed to solve: failed to parse error response 400: Our services aren't available right now +``` + +This error typically occurs when trying to export build cache to GitHub Actions cache (`type=gha`) while using Depot builders. Depot builds automatically enable layer caching. You don't need to export cache to GitHub Actions cache, and attempting to do so can cause conflicts. + +### How to resolve + +Remove both `--cache-from` and `--cache-to` from your build configuration: + +```bash +# Remove these flags: +depot build \ + --cache-from type=gha \ + --cache-to type=gha \ + . + +# Use this instead (Depot handles caching automatically): +depot build . +``` + +Once removed, your builds will use Depot's native caching, which is faster and more reliable than GitHub Actions cache. + +If you continue seeing this error after removing the cache configurations, [reach out to support](/help) with your project ID and build details. + +## Error: `failed to mount /tmp/buildkit-mount` + +If you see an error message like: + +```text +Error: failed to mount /tmp/buildkit-mountXXXXXXX: [{Type:overlay Source:overlay Target: Options:[lowerdir=/b/runc-stargz/snapshots/snapshotter/snapshots/XXXXX/fs +``` + +This indicates that BuildKit's snapshot manager cannot properly mount an overlay filesystem layer. + +This error commonly occurs when: + +- Cache layers become corrupted or inconsistent +- Snapshot metadata is out of sync with the actual filesystem state +- Previous builds left the cache in an inconsistent state +- Storage backend issues affect the overlay filesystem + +### How to resolve + +Reset your project's build cache to clear the corrupted layers: + +1. Navigate to your [Depot Dashboard](https://depot.dev) +2. Go to your project settings +3. Locate the **Cache Management** section +4. Click **Reset Cache** or **Clear Build Cache** +5. Confirm the cache reset operation +6. Retry your container build + +After resetting the cache, your build should complete successfully. The first build after a cache reset may take slightly longer as the cache rebuilds. + +If the error persists after resetting the cache, [reach out to support](/help) with: + +- Your project ID +- The full error message from your build logs +- Whether this happens consistently or intermittently + +## Error: `401 Unauthorized` during Docker pull + +If you encounter an error during container builds similar to: + +```text +Error: failed to solve: debian:trixie-slim: failed to resolve source metadata for http://docker.io/library/debian:trixie-slim: unexpected status from HEAD request to https://registry-1.docker.io/v2/library/debian/manifests/trixie-slim: 401 Unauthorized +``` + +This error typically indicates an issue with accessing Docker Hub. + +### How to resolve + +This error can occur due to Docker Hub outages, rate limiting, or authentication issues. Try these solutions: + +**1. Check Docker Hub status** + +First, check if Docker Hub is experiencing an outage or service disruption by visiting: [Docker's official status page](https://status.docker.com/) + +If Docker Hub is experiencing issues, you can continue your workflow by temporarily switching to AWS's public Docker mirror (see option 2 below). + +**2. Switch to AWS Docker Mirror** + +1. Identify the Docker image you need. For example, if you are using the Ubuntu image, the typical Docker Hub path would be `docker.io/library/ubuntu:latest`. +2. Replace the Docker Hub path with AWS's Docker mirror path. For Ubuntu, use: `public.ecr.aws/docker/library/ubuntu:latest`. +3. Update your Dockerfile or Docker commands to pull from the AWS mirror: + +```dockerfile +# Instead of: +FROM ubuntu:latest + +# Use: +FROM public.ecr.aws/docker/library/ubuntu:latest +``` + +Once Docker Hub is back online, you can switch back to the standard Docker Hub paths. + +**3. Authenticate with Docker Hub for higher rate limits** + +If you're hitting Docker Hub rate limits, you can authenticate with a Docker Hub account to increase your pull limits. Free Docker Hub accounts get higher limits than anonymous pulls, and paid accounts get even higher limits. + +To authenticate, create a Docker Hub account if you don't have one, then set up authentication in your build environment. + +## Error: `.git directory not found in build context` + +When using Depot's `build-push-action` for Docker builds, you might encounter an error such as: + +```text +Error: "/.git/refs/heads": not found. Please check if the files exist in the context. +``` + +By default, BuildKit does not include the `.git` directory in the build context, and uses the `git://` protocol instead. This can cause issues if your build process needs access to git information (for example, to determine commit hashes or branch names). + +### How to resolve + +Set the `BUILDKIT_CONTEXT_KEEP_GIT_DIR=1` build argument to tell BuildKit to keep the git repository in the context: + +```yaml +jobs: + build: + runs-on: ubuntu-latest + permissions: + contents: read + id-token: write + packages: write + steps: + - name: Check out + uses: actions/checkout@v4 + with: + fetch-depth: 2 + + - name: Set up Depot CLI + uses: depot/setup-action@v1 + + - name: Build and push container image + uses: depot/build-push-action@v1 + with: + project: your_project_id + push: true + platforms: linux/arm64,linux/amd64 + build-args: | + COMMIT_HASH=${{ github.sha }} + BUILDKIT_CONTEXT_KEEP_GIT_DIR=1 +``` + +For more information, refer to the [Docker documentation on keeping the git directory in the build context](https://docs.docker.com/build/building/context/#keep-git-directory). + +If you continue to see git-related errors after adding this build argument, verify that your checkout step is fetching the necessary git history and [reach out to support](/help) if needed. + +## Build hangs or builder won't start + +If your build is hung or a builder isn't coming online to serve build requests, this may be caused by: + +- A deadlock in BuildKit +- A builder that isn't coming online to serve the build request +- Build cache is full and needs to be cleared + +If you see any of these issues, you can reset the build cache for a project. + +### How to resolve + +Resetting the build cache purges the cache volume and launches a new build machine with a clean slate: 1. Go to the project's `Settings` page 2. Click the `Reset build cache` button at the bottom ## Multi-platform/multi-architecture image has a 3rd image with platform `unknown/unknown` -Docker introduced a new [provenance feature](https://docs.docker.com/build/attestations/slsa-provenance/) that tracks some info about the build itself, and it's implemented by attaching the data to the final image "manifest list". Many registries like GitHub Container Registry display the provenance data as an `unknown/unknown` image architecture. If you don't care about provenance or want a cleaner list in your registry, you can disable provenance during your image build. +Docker introduced a new [provenance feature](https://docs.docker.com/build/attestations/slsa-provenance/) that tracks some info about the build itself, and it's implemented by attaching the data to the final image "manifest list". Many registries like GitHub Container Registry display the provenance data as an `unknown/unknown` image architecture. + +### How to resolve + +If you don't care about provenance or want a cleaner list in your registry, you can disable provenance during your image build: ```bash depot build --provenance false ``` -### Disabling provenance when using `depot/build-push-action` or `depot/bake-action` +**When using `depot/build-push-action` or `depot/bake-action`:** -You can set `provenance` to `false` in your workflow step to disable provenance. +You can set `provenance` to `false` in your workflow step to disable provenance: ```yaml - uses: depot/build-push-action@v1 @@ -36,3 +247,31 @@ You can set `provenance` to `false` in your workflow step to disable provenance. provenance: false ... ``` + +## Cannot pull from private registry during build + +When building container images that need to pull from private registries (like in a `FROM` statement), you may need to provide authentication credentials to Depot. + +### How to resolve + +The `depot` CLI automatically uses your local Docker credentials provider. Any registry you've logged into with `docker login` is available when running a Depot build. + +For example, if your Dockerfile references a private registry: + +```dockerfile +FROM my-private-registry/project/image:version +... +``` + +Ensure you're logged into the registry from the machine where you're running `depot build`: + +```bash +docker login my-private-registry +depot build . +``` + +If you're still experiencing authentication issues: + +1. Confirm you're logged into the registry on the machine running `depot build` +2. Test that you can pull the image directly: `docker pull my-private-registry/project/image:version` +3. If the pull succeeds but the build fails, [reach out to support](/help) with your project ID and build details diff --git a/content/github-actions/quickstart.mdx b/content/github-actions/quickstart.mdx index 54ae84e..82415aa 100644 --- a/content/github-actions/quickstart.mdx +++ b/content/github-actions/quickstart.mdx @@ -44,8 +44,8 @@ To configure your GitHub Actions to use Depot runners, specify the runner label jobs: build: name: Build -- runs-on: ubuntu-22.04 -+ runs-on: depot-ubuntu-22.04 +- runs-on: ubuntu-24.04 ++ runs-on: depot-ubuntu-24.04 steps: ... ``` diff --git a/content/github-actions/runner-types.mdx b/content/github-actions/runner-types.mdx index de94891..63e9123 100644 --- a/content/github-actions/runner-types.mdx +++ b/content/github-actions/runner-types.mdx @@ -1,5 +1,5 @@ --- -title: GitHub Actions Runner Types +title: GitHub Actions Runner types ogTitle: Types of Depot-managed GitHub Action Runners description: Depot offers several different types of GitHub Actions runners, depending on your CI job needs. --- @@ -12,7 +12,7 @@ jobs: runs-on: depot-ubuntu-24.04 ``` -**Note**: We support the depot-ubuntu-latest alias for depot-ubuntu-24.04 if you prefer to use an evergreen Ubuntu version. +**Note**: We support the `depot-ubuntu-latest` alias for the current Ubuntu Long-Term Support (LTS) release if you prefer to use an evergreen version. **In-memory Disk Accelerator**: diff --git a/content/github-actions/troubleshooting.mdx b/content/github-actions/troubleshooting.mdx new file mode 100644 index 0000000..1eb57dc --- /dev/null +++ b/content/github-actions/troubleshooting.mdx @@ -0,0 +1,261 @@ +--- +title: Troubleshooting GitHub Actions runners +ogTitle: Troubleshoot common problems with Depot GitHub Actions runners +description: Solutions for common issues when using Depot-hosted GitHub Actions runners. +--- + +This guide provides troubleshooting steps for common issues when using Depot-hosted GitHub Actions runners. + +## Runner startup time + +When you start a new GitHub job using Depot GitHub Actions runners, you'll notice a brief delay before the runner is ready. Expect typical startup times to range from 10 to 45 seconds. + +The startup process involves two main steps: + +- **Provision AWS hardware (approximately 15 seconds):** Depot provisions fresh compute instances for each job with significant optimizations to the AWS startup process and pre-caching. +- **Register the GitHub Actions runner (variable time):** GitHub's control plane needs to register, authorize, and connect the runner. This timing varies based on GitHub's current load and rate limiting. + +Depot typically expects GitHub's control plane to take less than 1 second to send a job request to Depot API. Depot then allocates AWS hardware in approximately 15 seconds. After that, GitHub usually connects to the runner in under 7 seconds, and sends the job to the runner in under a second. + +You can check the detailed timing breakdown for the last jobs in your Depot dashboard in the graph at the top of the [GitHub Actions](/orgs/_/github-actions) page. + +Depot bills only for the time after the job has started running (tracked by the second), not provisioning time. + +### When to contact support + +If you consistently see allocation times longer than 1 minute during normal operation, reach out to support for investigation. + +## Workflows not starting on initial configuration + +If your GitHub Actions workflow remains in a queued state when first setting up Depot runners, the runner likely can't be allocated to your job. + +### How to resolve + +#### Check your runner labels + +Use a single runner label in your workflow configuration. While GitHub Actions supports specifying multiple labels in an array, this can cause stability issues. + +Correct usage: + +```yaml +runs-on: depot-ubuntu-24.04-4 +``` + +Avoid using multiple labels: + +```yaml +# Do *not* do this: +runs-on: [self-hosted, depot-ubuntu-24.04] +``` + +#### Verify repository access + +Depot runners register in the "default runner group" in GitHub. Check the repository access settings for the runner group to ensure your repo can use these runners: + +1. Go to your GitHub organization settings +2. Navigate to Actions > Runner groups +3. Select the "Default" runner group +4. Verify the allowed repositories list includes your repository. For public repositories, enable **Allow public repositories**. + +**Note:** If you've configured the [Selected workflows](https://docs.github.com/en/enterprise-cloud@latest/actions/how-tos/manage-runners/self-hosted-runners/manage-access#changing-which-workflows-can-access-a-runner-group) filter for your runner group, an incorrect workflow reference pattern can prevent jobs from starting even when runners are available. + +#### Confirm app permissions + +The "Depot Managed Runners" GitHub app must have permission to see your repo. You can confirm your app is active in the [organization settings](https://depot.dev/orgs/_/settings) under **GitHub Actions Runners** + +If your workflow still gets stuck after checking these settings, reach out to support with your workflow run URL and repository name for investigation of the runner allocation issue. + +## Jobs appear as long-running in the dashboard + +Sometimes jobs may appear as long-running in the Depot dashboard even though they've completed on GitHub. This happens when GitHub doesn't correctly inform Depot that a job has finished. + +**Important:** Billing isn't affected by stale jobs. Depot only bills for the time the job actually runs on a runner instance. + +Stale jobs eventually time out on their own. However, if you need immediate resolution, Depot can manually run a resync with GitHub to clear the job and update the UI to reflect the correct status. + +### How to resolve + +Check the job status directly in your GitHub Actions tab to confirm its cancellation or completion. + +If the job remains stale in the Depot dashboard and you need it cleared immediately, [reach out to support](/help) with: + +- Your organization name +- The repository and workflow run URL +- The job name or ID that appears stale + +The support team will trigger a manual resync with GitHub to update the job status. + +## Error: `Failed to write to Depot job log - No space left on device` + +This error occurs when your build process needs more disk space than your runner has available. + +Each Depot runner type has a set amount of disk space. The OS and pre-installed software consumes approximately 70 GB, and the remaining space is available for your builds. For runner specifications, see [Depot Runner Types](/docs/github-actions/runner-types). + +### How to resolve + +#### Upgrade to a larger runner size + +To use a runner with more disk capacity, specify a larger runner type in your workflow configuration: + +```yaml +runs-on: depot-ubuntu-24.04-8 # 8 vCPU, more disk space +``` + +If you continue to hit disk space limits even on larger runner sizes, [reach out to support](/help) with your workflow details for help optimizing your build process or discussing custom runner configurations. + +#### Disk space cleanup + +Clean up unnecessary files before and during your build process. For example, you can remove temporary files and clean package caches on Linux workflows: + +```yaml +- name: Clean up disk space + run: | + rm -rf /tmp/* + sudo apt-get clean +``` + +#### Use disk cleanup actions + +Several GitHub Actions can help manage disk space automatically. Popular options include actions that remove unused tools, or clear system caches. + +## Error: `The self-hosted runner lost communication with the server` + +The following error typically indicates a temporary connectivity issue between the Depot runner and GitHub's servers: + +```text +The self-hosted runner lost communication with the server. Verify the machine is running and has a healthy network connection. Anything in your workflow that terminates the runner process, starves it for CPU/Memory, or blocks its network access can cause this error. +``` + +### How to resolve + +First, check [status.depot.dev](https://status.depot.dev) for any reported outages. If Depot is experiencing an incident, monitor the status page and wait for the resolution. Once the issue resolves, retry your workflow run. + +Next, check the CPU and memory usage of your build process. Open the job details in your Depot dashboard and look at the resource usage graphs at the top of the page. If your build consumes excessive CPU or memory, consider optimizing your build steps to reduce resource consumption. + +If you see this error frequently and the resource usage graphs show no issues, reach out to support with your workflow run ID and the approximate time the error occurred for investigation of specific connectivity issues affecting your builds. + +## Error: `The operation was canceled.` + +This error indicates that GitHub terminated your job before it completed. Some common reasons for workflow cancellation include manual cancellation, workflow concurrency settings, or memory exhaustion. + +### How to resolve + +#### Manual cancellation + +Manual cancellation through the GitHub UI causes this error most commonly. Check your workflow run history to confirm whether someone clicked the cancel button for this job or workflow run. + +#### Workflow concurrency settings + +If you set `cancel-in-progress: true` in your workflow's concurrency settings, GitHub automatically cancels running workflows when a new run starts: + +```yaml +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true +``` + +If you're seeing unexpected cancellations, review your concurrency configuration to ensure it matches your desired behavior. + +#### Memory exhaustion + +Memory exhaustion on the runner can also cause this error. To investigate: + +1. Open your job details page in the Depot dashboard +2. Review the memory usage graph at the top of the page +3. Check if memory consumption reached or exceeded the available capacity + +If memory usage consistently runs high or maxes out: + +- Upgrade to a larger runner type with more memory capacity. See [Depot Runner Types](/docs/github-actions/runner-types) for available options. +- Optimize your build process to reduce memory consumption +- Split large jobs into smaller, parallel jobs + +If you've verified these causes and the issue persists, [reach out to support](/help) with your workflow run URL and job details for further investigation. + +## Unable to cancel GitHub Actions workflow + +Unrelated to Depot, GitHub Actions workflows sometimes become stuck and don't respond to normal cancel requests through the GitHub UI. When this happens, you can use the [GitHub REST API to force cancel the workflow run](https://docs.github.com/en/rest/actions/workflow-runs#force-cancel-a-workflow-run). + +### How to resolve + +Find the run ID of the stuck workflow from your GitHub Actions page. It's in the URL: `github.com/{owner}/{repo}/actions/runs/{run_id}` + +After obtaining the run ID, you can use the GitHub API to attempt a regular cancel: + +```bash +curl -X POST \ + -H "Accept: application/vnd.github+json" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + https://api.github.com/repos/{owner}/{repo}/actions/runs/{run_id}/cancel +``` + +If the workflow doesn't respond, use the force cancel option: + +```bash +curl -X POST \ + -H "Accept: application/vnd.github+json" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + https://api.github.com/repos/{owner}/{repo}/actions/runs/{run_id}/force-cancel +``` + +**Important:** Use force cancel only when necessary, as it bypasses normal workflow conditions and cleanup steps. + +For more details, see [GitHub's official REST API documentation](https://docs.github.com/en/rest/actions/workflow-runs). + +If you frequently encounter stuck workflows with Depot runners specifically, reach out to support for investigation of potential underlying issues. + +## SSH access to Depot GitHub Actions runners + +Sometimes you may need to inspect the running environment to diagnose an issue. Depot GitHub Actions runners are compatible with [action-tmate](https://github.com/mxschmitt/action-tmate), a GitHub Action that allows you to create an SSH session during your workflow run. + +### How to set up SSH access + +#### Add `action-tmate` to your workflow + +Integrate `action-tmate` into your CI pipeline by adding it as a step in your workflow file: + +```yaml +steps: + - uses: actions/checkout@v4 + + - name: Setup tmate session + uses: mxschmitt/action-tmate@v3 + + - name: Run tests + run: npm test +``` + +#### Connect to your runner + +When your job reaches the `action-tmate` step, open the Checks tab in your Pull Request and scroll to the bottom. You can connect either directly per SSH or via a web based terminal. + +#### Debug and exit + +Inspect the environment, run commands, and debug as needed. Once done, you can exit the SSH session and let the CI job continue. + +For additional information and usage scenarios, visit the [action-tmate GitHub page](https://github.com/mxschmitt/action-tmate). + +**Note:** Depot doesn't maintain the `action-tmate` action. Please report any issues or problems on the [project GitHub page](https://github.com/mxschmitt/action-tmate/issues). + +## Delayed allocation for macOS runners + +macOS build jobs may experience longer wait times to start processing when demand is higher than normal. + +Apple's licensing requires a minimum 24-hour lease for macOS instances, which results in the following constraints: + +- **No autoscaling:** Unlike Linux and Windows runners that can spin up and down based on demand, macOS instances can't be dynamically provisioned. +- **Fixed pool size:** Depot maintains a constant number of macOS runners that operate continuously. +- **First In, First Out (FIFO) processing:** Jobs process in the order they arrive. +- **Jobs queue if demand spikes unexpectedly:** When demand exceeds the fixed macOS runner capacity, jobs enter a queue and wait for the next available runner. + +### Dedicated macOS runners + +For organizations requiring more predictable build times, Depot offers the option to reserve dedicated macOS runner instances exclusively for your organization. Your jobs run immediately on your dedicated runners when available. If all your dedicated runners are busy processing jobs, additional jobs can still fall back to the shared pool. + +Dedicated runners require an annual commitment for fixed capacity tailored to your concurrent job needs. + +[Contact our support team](/help) to learn more about dedicated macOS runners and pricing. + +## Additional support + +If you can't find a solution in this guide or the [Depot GitHub Actions documentation](/docs/github-actions/overview), [reach out to support](/help). diff --git a/content/overview/account.mdx b/content/overview/account.mdx new file mode 100644 index 0000000..913a26a --- /dev/null +++ b/content/overview/account.mdx @@ -0,0 +1,77 @@ +--- +title: Manage your Depot account and organizations +ogTitle: Manage your Depot account and organizations +description: Learn how your Depot account works and how to manage it. +--- + +Your Depot account is your login and identity on the Depot platform. Your account can be an owner or member of multiple Depot organizations. + +The [Depot dashboard](/orgs/) is where you manage your account and organizations in Depot. + +## Account settings + +You can access your user account settings in the Depot dashboard. Click the organization name at the top of the page and select **User settings**. + +![Select User settings from the depot dashboard menu](/images/docs/dashboard-user-settings.png) + +From the user [Settings](/settings) page you can do the following: + +- Sign out of your Depot account +- Update your profile name +- Create [user access tokens](/docs/cli/authentication#user-access-tokens) to authenticate for local development +- Add public SSH keys for authentication + +You can't change the email address for your Depot account. To delete an account, please send a request to help@depot.dev. + +## Organizations + +Every action you take in Depot is within the context of an organization. An organization typically represents a single company or team. Depot bills at the organization level, consolidating all usage across products into a single invoice. + +You can view all of your builds, runners, usage, and billing per organization in your Depot dashboard. Configure organization-level settings in the organization [Settings](/orgs/_/settings) page. + +### Create an organization + +When you sign up for Depot, we either prompt you to create an organization, or you automatically join the organization you were invited to. + +To create a new organization or request to join an organization: + +1. Log in to your [Depot dashboard](/orgs). +2. Click the organization name at the top of the page. +3. Go to **Switch organization** and select **Create a new organization**. + ![Select Create new organization from the depot dashboard menu](/images/docs/dashboard-create-new-org.png) + +### Switch organizations in the Depot dashboard + +1. Log in to your [Depot dashboard](/orgs). +2. Click the organization name at the top of the page. +3. Go to **Switch organization** and select another organization from the list. + +### Organization access roles + +Organizations have role-based access. When you create an organization, you have the owner role and permissions. + +| Role | Permissions | +| ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Owner | Manage all organization settings, such as billing, payments, SSO, audit logging, tokens, and GitHub connections. Manage members, including invitations, roles, and removal. Manage projects, builds, and runners. Access usage and analytics. | +| Member | Manage projects, builds, runners. Access usage and analytics. | + +### Invite and manage organization members + +If you have a Startup or Business plan, you can invite users to join an organization. + +1. Log in to your [Depot dashboard](/orgs). +2. Under **Organization**, click [Settings](/orgs/_/settings). +3. From the **Members** section of the **Settings** page you can: + - View members and change their role. + - View or remove pending invites. + - Invite new members using either their existing Depot account email or the email they'll use to sign up for Depot. + +When you invite someone to join your organization, they receive an email from `help@depot.dev` with an **Accept invite** link. If the user doesn't have a Depot account, or isn't signed in, the **Accept invite** link redirects them to sign in or sign up. After the user authenticates, they become an organization member and can access the organization in the Depot dashboard. + +### Delete an organization + +To delete an organization, please send a request to help@depot.dev. Include the name of the organization you want to delete. We'll process the deletion manually to ensure all associated projects and runners are properly removed. + +## Billing + +Each Depot organization has its own [plan](/pricing) and billing. To view an organization's billing, payment, and plan details, go to the organization [Settings](/orgs/_/settings) page in your Depot dashboard. For billing issues, see our [troubleshooting page](/docs/troubleshooting#billing). diff --git a/content/overview/faq.mdx b/content/overview/faq.mdx index ec18375..8694910 100644 --- a/content/overview/faq.mdx +++ b/content/overview/faq.mdx @@ -98,13 +98,13 @@ To use Depot's GitHub Actions runners, you need to: For example, change: ```yaml -runs-on: ubuntu-22.04 +runs-on: ubuntu-24.04 ``` to: ```yaml -runs-on: depot-ubuntu-22.04 +runs-on: depot-ubuntu-24.04 ``` ### What runner types does Depot offer? diff --git a/content/overview/index.mdx b/content/overview/index.mdx index 147662b..6e95b9b 100644 --- a/content/overview/index.mdx +++ b/content/overview/index.mdx @@ -1,13 +1,14 @@ --- title: Get started with Depot ogTitle: Get started with Depot -description: Depot accelerates your most important developer workflows. +description: Depot accelerates your most important development workflows. hideToc: true --- import {DocsCard, DocsCardGrid} from '~/components/docs/DocsCard' import {DocsCTA, DocsCTASecondary} from '~/components/blog/CTA' import {TrackedLink} from '~/components/TrackedLink' +import {CodeIcon, CpuIcon, DatabaseIcon, GitHubLogoIcon, RobotIcon, ShippingContainerIcon} from '~/components/icons' Depot accelerates your most important development workflows. @@ -19,6 +20,7 @@ Depot accelerates your most important development workflows. } links={[ {text: 'Quickstart: Build Docker images faster', href: '/docs/container-builds/quickstart'}, {text: 'Learn more about Depot container builds', href: '/docs/container-builds/overview'}, @@ -27,24 +29,31 @@ Depot accelerates your most important development workflows. } links={[ {text: 'Quickstart: Use fast runners for your GitHub Actions', href: '/docs/github-actions/quickstart'}, {text: 'Learn more about Depot GitHub Actions runners', href: '/docs/github-actions/overview'}, ]} /> - +} + links={[{text: 'Learn more about Depot Cache', href: '/docs/cache/overview'}]} +/> } links={[ {text: 'Quickstart: Run Claude Code in a sandbox', href: '/docs/agents/claude-code/quickstart'}, - {text: 'Learn more about Depot remote agent sandboxes', href: '/docs/agents/overview'}, + {text: 'Learn more about Depot agent sandboxes', href: '/docs/agents/overview'}, ]} /> } links={[ {text: 'Quickstart: Use our container image registry', href: '/docs/registry/quickstart'}, {text: 'Learn more about the Depot container registry', href: '/docs/registry/overview'}, @@ -53,10 +62,11 @@ Depot accelerates your most important development workflows. } links={[ { text: "Reference: Access Depot's underlying architecture programmatically", - href: '/docs/container-builds/reference/api-overview', + href: '/docs/api/overview', }, ]} /> @@ -66,7 +76,7 @@ Depot accelerates your most important development workflows. - Deploy the entire Depot data plane in your AWS account. Learn more about Depot Managed → + Deploy the entire Depot data plane in your AWS account with Depot Managed → diff --git a/content/overview/security.mdx b/content/overview/security.mdx index 337b6ef..e603d4f 100644 --- a/content/overview/security.mdx +++ b/content/overview/security.mdx @@ -71,3 +71,17 @@ In addition to users, Depot also allows creating trust relationships with GitHub ### Caching and Builder Access Access to create project builds effectively equates to access to the builder VM due to the nature of how `docker build` works. Anyone with access to build a project can access that project's build cache files and potentially add, edit, or remove cache entries. You should be careful that you trust the users and trust relationships that you have given access to a project and use tools like OIDC trust relationships to limit access to only the necessary scope. + +## Understanding security and isolation in GitHub Actions runners + +Each GitHub Actions job runs on a new EC2 instance that has never been used before, ensuring complete isolation at the hardware level. Once the job completes, we destroy the instance, maintaining a single tenant environment for each job. + +### Caching and storage + +GitHub Actions can integrate with Depot Cache, allowing you to store data in a cache service backed by S3. You have control over what gets stored in the cache, as you choose which items to mark for caching. + +Cache entries stored in the Depot Cache are scoped by repository. Cache entries can only be read by the same repository that saved them. This prevents repositories from having key collisions when using the same cache key and ensures that one repository cannot unexpectedly read cache entries from another repository of a different trust level (like reading from or writing to a private repository from a public repository). + +### Handling secrets + +The same masking rules that apply in GitHub Actions are enforced in the runner environment. Any secrets you configure will be masked in the logs, preventing exposure. However, avoid hardcoding secrets in your codebase to prevent potential security breaches. diff --git a/content/overview/troubleshooting.mdx b/content/overview/troubleshooting.mdx new file mode 100644 index 0000000..b4d3b60 --- /dev/null +++ b/content/overview/troubleshooting.mdx @@ -0,0 +1,40 @@ +--- +title: Troubleshooting +ogTitle: How to troubleshoot common problems using Depot +description: Overview of common troubleshooting steps for using Depot. +--- + +This page provides an overview of troubleshooting resources for Depot products. + +## Container builds + +If you're having issues with Docker image builds using Depot, see the [container builds troubleshooting guide](/docs/container-builds/troubleshooting). + +## GitHub Actions runners + +If you're having issues with Depot-hosted GitHub Actions runners, see the [GitHub Actions troubleshooting guide](/docs/github-actions/troubleshooting). + +## Billing + +### Payment failures and retries + +Payment failures usually occur because of insufficient funds, expired cards, or temporary issues with your payment provider. + +When a payment fails, we automatically retry the charge over the course of 14 days. You'll receive an email notification after each failed attempt. + +**How to resolve** + +If you receive a payment failure notification: + +1. Check that your payment method has sufficient funds +2. Verify that your card hasn't expired +3. Update your payment information through the link in the payment failure email +4. Contact your bank if you suspect the charge was blocked + +If all retry attempts fail, your subscription will be automatically canceled to prevent further charges. You can reactivate your subscription at any time by updating your payment method and contacting our support team. + +If you're experiencing repeated payment failures or need help updating your payment information, reach out to support and we'll help get your subscription sorted. + +## Additional support + +If you can't find a solution in these guides or the product documentation, [reach out to our support team](/help). diff --git a/content/registry/quickstart.mdx b/content/registry/quickstart.mdx index 6f02e96..ca079a5 100644 --- a/content/registry/quickstart.mdx +++ b/content/registry/quickstart.mdx @@ -99,7 +99,7 @@ depot pull --project my-image To pull a build from the Depot Registry in a Kubernetes cluster, you can use the `kubectl` command to [create a secret with the Docker registry credentials](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/), then create a Kubernetes deployment that uses the secret to pull the image. ```shell -kubectl create secret depot-registry regcred \ +kubectl create secret docker-registry regcred \ --docker-server=registry.depot.dev \ --docker-username=x-token \ --docker-password=