Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docs/concepts/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,3 +89,12 @@ Here are the different ways sensitive config values are stored depending on the
:::info
Please note that while Defang supports setting sensitive config, it does not support the [`secrets`](https://docs.docker.com/reference/compose-file/secrets/) top-level element as seen in the Compose specification. Please see our [Compose](/docs/concepts/compose) page for more details.
:::

## Supported Providers

| Provider | Config Support |
|----------------|:--------------:|
| Playground | ✅ |
| AWS | ✅ |
| DigitalOcean | ✅ |
| GCP | ✅ |
9 changes: 9 additions & 0 deletions docs/concepts/estimation.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,3 +64,12 @@ Now that you have estimated the costs associated with your project. You are read
defang compose up [--provider aws|gcp|digitalocean] [--mode affordable|balanced|high_availability]
```

## Supported Providers

| Provider | Estimation Support |
|----------------|:------------------:|
| Playground | ❌ |
| AWS | ✅ |
| DigitalOcean | ❌ |
| GCP | ✅ |

150 changes: 150 additions & 0 deletions docs/concepts/one_off_jobs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
---
title: One-off Jobs
description: Defang enables you to run one-off jobs during your deployment workflow.
sidebar_position: 280
---

# One-off Jobs
Defang enables you to run one-off jobs during your deployment workflow. One-off jobs are commands that run at specific points in the deployment process, such as after your database is ready but before your application starts.

One-off jobs are run a single time, and failure to run a one-off job will cause the entire deployment to fail.

:::info
Currently, AWS and GCP are supported for one-off jobs. Support for Digital Ocean is coming soon.
:::

## When should one-off jobs be used?
One-off jobs are useful for running commands that need to be executed before your application starts. Common use cases include:

- Database migrations
- Warming or cleaning caches
- Running build scripts

## How to configure one-off jobs

One-off jobs can be configured in your `compose.yaml` file as services with a [Restart Policy](https://docs.docker.com/reference/compose-file/services/#restart) set to `"no"`.

In the example below, we define a one-off job called `release` that runs a script called `release.sh`. This job will run once during the deployment process. It will be run concurrently with the `web` service.

<table width="100%">
<thead>
<tr>
<th><code>compose.yaml</code></th>
<th>Workflow</th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top">

```yaml
services:
app:
build: .
ports:
- "80:80"

release:
build: .
command: ["./release.sh"]
restart: "no"
```

</td>
<td valign="top">

```mermaid
flowchart LR
subgraph workspace["Deployment Workflow"]
deployment_start
app
release
deployment_complete

deployment_start --> app
deployment_start --> release
app --> deployment_complete
release --> deployment_complete
end
```

</td>
</tr>
</tbody>
</table>

### How to specify job dependencies

You can specify when the job should run by using [`depends_on`](https://docs.docker.com/reference/compose-file/services/#depends_on) to define dependencies on other services. For example, when running database migrations, you would typically want to ensure that the database service is ready before executing the migration job.

Here is an example configuration for a one-off job that runs database migrations after the database service is ready:

<table width="100%">
<thead>
<tr>
<th><code>compose.yaml</code></th>
<th>Workflow</th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top">

```yaml
services:
db:
image: postgres:latest

migrate:
build: .
command: ["./migrate.sh"]
depends_on:
- db
restart: "no"

web:
build: .
ports:
- "80:80"
depends_on:
- migrate
```

</td>
<td valign="top">

```mermaid
flowchart LR
subgraph workspace["Deployment Workflow"]
deployment_start
db
web
migrate
deployment_complete

deployment_start --> db
db --> migrate
migrate --> web
web --> deployment_complete
end
```

</td>
</tr>
</tbody>
</table>

## How do one-off jobs work under the hood?

One off jobs are deployed as temporary containers on the same infrastructure as your main services. They are deployed as separate containers to ensure that they do not interfere with the main application services, so resources can be configured independently. They will share the same network and security groups as your other services, allowing them to communicate with your application and database services as needed.

### Supported Providers

| Provider | Release Task Support |
|----------------|:--------------------:|
| Playground | ✅ |
| AWS | ✅ |
| DigitalOcean | ❌ |
| GCP | ✅ |


9 changes: 9 additions & 0 deletions docs/concepts/railpack.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,3 +144,12 @@ services:
reservations:
memory: 512M
```

## Supported Providers

| Provider | Railpack Support |
|----------------|:----------------:|
| Playground | ❌ |
| AWS | ✅ |
| DigitalOcean | ❌ |
| GCP | ✅ |
2 changes: 1 addition & 1 deletion docs/concepts/resources.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ Defang uses `shm_size` to configure both the memory and disk space available to

The default `shm_size` values for each platform are as follows. More or less may be specified.

| Platform | `shm_size` Minimum |
| Provider | `shm_size` Minimum |
| ------------- | ------------------ |
| AWS | 16G |
| Digital Ocean | 8G |
Expand Down
6 changes: 3 additions & 3 deletions docs/concepts/scaling.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,10 @@ Auto-scaling systems typically rely on:
- If average CPU > 85% for 5 minutes → scale up by 2 instances.
- **Cooldown Periods**: Delays between scaling events to prevent rapid, repeated changes (flapping).

### Supported Platforms
### Supported Providers

| Platform | Auto-Scaling Support |
|----------------|:----------------------:|
| Provider | Auto-Scaling Support |
|----------------|:--------------------:|
| Playground | ❌ |
| AWS | ✅ |
| DigitalOcean | ❌ |
Expand Down
85 changes: 82 additions & 3 deletions docs/tutorials/migrating-from-heroku.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ This tutorial will guide you through the process of migrating your Heroku applic
* [The Heroku CLI (optional, but recommended)](https://devcenter.heroku.com/articles/heroku-cli#install-the-heroku-cli)
* Cloud Account Credentials
* [AWS](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-authentication.html)
* [GCP](https://cloud.google.com/docs/authentication/set-up-adc-local-dev-environment) (coming soon)
* [GCP](https://cloud.google.com/docs/authentication/set-up-adc-local-dev-environment)

:::tip
**Do I need a Dockerfile?**
Expand Down Expand Up @@ -167,7 +167,7 @@ Now all you need to do is deploy your application to the cloud.

### Deploying to AWS

If you're deploying to AWS, you'll need to invoke `defang compose up` with your AWS access credentials in the environment:
If you're deploying to AWS, you'll need to invoke `defang compose up --provider aws` with your AWS access credentials in the environment:

<details>
<summary>
Expand Down Expand Up @@ -285,7 +285,86 @@ See our full tutorial on [deploying to AWS](/docs/tutorials/deploy-to-aws).

### Deploying to GCP

GCP support for deployments without Dockerfiles coming soon.
If you're deploying to GCP, you'll need to invoke `defang compose up --provider gcp` with your GCP access credentials in the environment:

<details>
<summary>
Run the following command
```
GCP_PROJECT_ID=my-project-123456 defang compose up --provider gcp
```
</summary>
```
* Using Google Cloud Platform provider from command line flag
! Defang cannot monitor status of the following managed service(s): [postgres release].
To check if the managed service is up, check the status of the service which depends on it.
! service "postgres": missing memory reservation; using provider-specific defaults. Specify deploy.resources.reservations.memory to avoid out-of-memory errors
! service "web": ingress port without healthcheck defaults to GET / HTTP/1.1
* Packaging the project files for web at /Users/defang/wk/vast-badlands
* Uploading the project files for web
* Setting up defang CD in GCP project my-project-123456, this could take a few minutes
* Packaging the project files for release at /Users/defang/wk/vast-badlands
* Uploading the project files for release
* Tailing logs for deployment ID fgdh4ct0hopz ; press Ctrl+C to detach:
* Showing only build logs and runtime errors. Press V to toggle verbose mode.
2025-09-08T10:28:08.634-04:00 cd ** Update started for stack beta
2025-09-08T10:28:17.990-04:00 cd ** Updating service "postgres"
2025-09-08T10:28:44.773-04:00 cd ** Building image for "release"...
2025-09-08T10:28:58.012-04:00 cd ** Building image for "web"...
2025-09-08T10:47:02.479-04:00 release I, [2025-09-08T14:47:02.409371 #1] INFO -- : Migrating to CreateMembers (20240416182733)
2025-09-08T10:47:02.479-04:00 release == 20240416182733 CreateMembers: migrating ====================================
2025-09-08T10:47:02.479-04:00 release -- create_table(:members)
2025-09-08T10:47:02.479-04:00 release -> 0.0073s
2025-09-08T10:47:02.479-04:00 release == 20240416182733 CreateMembers: migrated (0.0073s) ===========================
2025-09-08T10:47:02.479-04:00 release I, [2025-09-08T14:47:02.432075 #1] INFO -- : Migrating to DeviseCreateUsers (20240417165503)
2025-09-08T10:47:02.479-04:00 release == 20240417165503 DeviseCreateUsers: migrating ================================
2025-09-08T10:47:02.479-04:00 release -- create_table(:users)
2025-09-08T10:47:02.479-04:00 release -> 0.0080s
2025-09-08T10:47:02.479-04:00 release -- add_index(:users, :email, {:unique=>true})
2025-09-08T10:47:02.479-04:00 release -> 0.0022s
2025-09-08T10:47:02.479-04:00 release -- add_index(:users, :reset_password_token, {:unique=>true})
2025-09-08T10:47:02.479-04:00 release -> 0.0021s
2025-09-08T10:47:02.479-04:00 release == 20240417165503 DeviseCreateUsers: migrated (0.0124s) =======================
2025-09-08T10:47:02.479-04:00 release I, [2025-09-08T14:47:02.458370 #1] INFO -- : Migrating to AddUserIdToMembers (20240417202202)
2025-09-08T10:47:02.479-04:00 release == 20240417202202 AddUserIdToMembers: migrating ===============================
2025-09-08T10:47:02.479-04:00 release -- add_column(:members, :user_id, :integer)
2025-09-08T10:47:02.479-04:00 release -> 0.0044s
2025-09-08T10:47:02.479-04:00 release -- add_index(:members, :user_id)
2025-09-08T10:47:02.479-04:00 release -> 0.0019s
2025-09-08T10:47:02.479-04:00 release == 20240417202202 AddUserIdToMembers: migrated (0.0064s) ======================
2025-09-08T10:48:16.157-04:00 web => Booting Puma
2025-09-08T10:48:16.157-04:00 web => Rails 7.1.3.2 application starting in production
2025-09-08T10:48:16.157-04:00 web => Run `bin/rails server --help` for more startup options
2025-09-08T10:48:18.625-04:00 web [1] Puma starting in cluster mode...
2025-09-08T10:48:18.625-04:00 web [1] * Puma version: 6.4.2 (ruby 3.3.4-p94) ("The Eagle of Durango")
2025-09-08T10:48:18.625-04:00 web [1] * Min threads: 5
2025-09-08T10:48:18.625-04:00 web [1] * Max threads: 5
2025-09-08T10:48:18.625-04:00 web [1] * Environment: production
2025-09-08T10:48:18.626-04:00 web [1] * Master PID: 1
2025-09-08T10:48:18.626-04:00 web [1] * Workers: 2
2025-09-08T10:48:18.626-04:00 web [1] * Restarts: (✔) hot (✔) phased
2025-09-08T10:48:18.626-04:00 web [1] * Listening on http://0.0.0.0:5000
2025-09-08T10:48:18.627-04:00 web [1] Use Ctrl-C to stop
```
</details>

### Deploying to DigitalOcean

If you're deploying to GCP, you'll need to invoke `defang compose up --provider digitalocean` with your DigitalOcean access credentials in the environment:

:::warning
Some Heroku applications may require some manual adjustments to be deployed to DigitalOcean with Defang.
Dockerfiles are required for deployments to DigitalOcean, so you may need to create one if your application does not already have one. Support for automatic Dockerfile generation is coming soon with [Railpack](/docs/concepts/railpack.md).
Defang also does not yet support [one-off jobs](/docs/concepts/one_off_jobs) on DigitalOcean, so you will need to modify your services to run these tasks during initialization.
:::

```
export DIGITALOCEAN_TOKEN=your_digitalocean_token
export SPACES_ACCESS_KEY_ID=your_spaces_access_key_id
export SPACES_SECRET_ACCESS_KEY=your_spaces_secret_access_key
$ defang compose up --provider digitalocean
```


## Step 3 - Migrating your data

Expand Down