diff --git a/docs/concepts/configuration.md b/docs/concepts/configuration.md index 17dd726ee..f80e8ecda 100644 --- a/docs/concepts/configuration.md +++ b/docs/concepts/configuration.md @@ -89,3 +89,12 @@ Here are the different ways sensitive config values are stored depending on the :::info Please note that while Defang supports setting sensitive config, it does not support the [`secrets`](https://docs.docker.com/reference/compose-file/secrets/) top-level element as seen in the Compose specification. Please see our [Compose](/docs/concepts/compose) page for more details. ::: + +## Supported Providers + +| Provider | Config Support | +|----------------|:--------------:| +| Playground | ✅ | +| AWS | ✅ | +| DigitalOcean | ✅ | +| GCP | ✅ | diff --git a/docs/concepts/estimation.md b/docs/concepts/estimation.md index ef1c80dd2..148aa250e 100644 --- a/docs/concepts/estimation.md +++ b/docs/concepts/estimation.md @@ -64,3 +64,12 @@ Now that you have estimated the costs associated with your project. You are read defang compose up [--provider aws|gcp|digitalocean] [--mode affordable|balanced|high_availability] ``` +## Supported Providers + +| Provider | Estimation Support | +|----------------|:------------------:| +| Playground | ❌ | +| AWS | ✅ | +| DigitalOcean | ❌ | +| GCP | ✅ | + diff --git a/docs/concepts/one_off_jobs.md b/docs/concepts/one_off_jobs.md new file mode 100644 index 000000000..755bcbab4 --- /dev/null +++ b/docs/concepts/one_off_jobs.md @@ -0,0 +1,150 @@ +--- +title: One-off Jobs +description: Defang enables you to run one-off jobs during your deployment workflow. +sidebar_position: 280 +--- + +# One-off Jobs +Defang enables you to run one-off jobs during your deployment workflow. One-off jobs are commands that run at specific points in the deployment process, such as after your database is ready but before your application starts. + +One-off jobs are run a single time, and failure to run a one-off job will cause the entire deployment to fail. + +:::info +Currently, AWS and GCP are supported for one-off jobs. Support for Digital Ocean is coming soon. +::: + +## When should one-off jobs be used? +One-off jobs are useful for running commands that need to be executed before your application starts. Common use cases include: + +- Database migrations +- Warming or cleaning caches +- Running build scripts + +## How to configure one-off jobs + +One-off jobs can be configured in your `compose.yaml` file as services with a [Restart Policy](https://docs.docker.com/reference/compose-file/services/#restart) set to `"no"`. + +In the example below, we define a one-off job called `release` that runs a script called `release.sh`. This job will run once during the deployment process. It will be run concurrently with the `web` service. + + + + + + + + + + + + + + +
compose.yamlWorkflow
+ +```yaml +services: + app: + build: . + ports: + - "80:80" + + release: + build: . + command: ["./release.sh"] + restart: "no" +``` + + + +```mermaid +flowchart LR + subgraph workspace["Deployment Workflow"] + deployment_start + app + release + deployment_complete + + deployment_start --> app + deployment_start --> release + app --> deployment_complete + release --> deployment_complete + end +``` + +
+ +### How to specify job dependencies + +You can specify when the job should run by using [`depends_on`](https://docs.docker.com/reference/compose-file/services/#depends_on) to define dependencies on other services. For example, when running database migrations, you would typically want to ensure that the database service is ready before executing the migration job. + +Here is an example configuration for a one-off job that runs database migrations after the database service is ready: + + + + + + + + + + + + + + +
compose.yamlWorkflow
+ +```yaml +services: + db: + image: postgres:latest + + migrate: + build: . + command: ["./migrate.sh"] + depends_on: + - db + restart: "no" + + web: + build: . + ports: + - "80:80" + depends_on: + - migrate +``` + + + +```mermaid +flowchart LR + subgraph workspace["Deployment Workflow"] + deployment_start + db + web + migrate + deployment_complete + + deployment_start --> db + db --> migrate + migrate --> web + web --> deployment_complete + end +``` + +
+ +## How do one-off jobs work under the hood? + +One off jobs are deployed as temporary containers on the same infrastructure as your main services. They are deployed as separate containers to ensure that they do not interfere with the main application services, so resources can be configured independently. They will share the same network and security groups as your other services, allowing them to communicate with your application and database services as needed. + +### Supported Providers + +| Provider | Release Task Support | +|----------------|:--------------------:| +| Playground | ✅ | +| AWS | ✅ | +| DigitalOcean | ❌ | +| GCP | ✅ | + + diff --git a/docs/concepts/railpack.md b/docs/concepts/railpack.md index af0a5b5c2..c8ce0d2f8 100644 --- a/docs/concepts/railpack.md +++ b/docs/concepts/railpack.md @@ -144,3 +144,12 @@ services: reservations: memory: 512M ``` + +## Supported Providers + +| Provider | Railpack Support | +|----------------|:----------------:| +| Playground | ❌ | +| AWS | ✅ | +| DigitalOcean | ❌ | +| GCP | ✅ | diff --git a/docs/concepts/resources.md b/docs/concepts/resources.md index f2982846e..19f239518 100644 --- a/docs/concepts/resources.md +++ b/docs/concepts/resources.md @@ -67,7 +67,7 @@ Defang uses `shm_size` to configure both the memory and disk space available to The default `shm_size` values for each platform are as follows. More or less may be specified. -| Platform | `shm_size` Minimum | +| Provider | `shm_size` Minimum | | ------------- | ------------------ | | AWS | 16G | | Digital Ocean | 8G | diff --git a/docs/concepts/scaling.md b/docs/concepts/scaling.md index 192a309e7..54f550851 100644 --- a/docs/concepts/scaling.md +++ b/docs/concepts/scaling.md @@ -65,10 +65,10 @@ Auto-scaling systems typically rely on: - If average CPU > 85% for 5 minutes → scale up by 2 instances. - **Cooldown Periods**: Delays between scaling events to prevent rapid, repeated changes (flapping). -### Supported Platforms +### Supported Providers -| Platform | Auto-Scaling Support | -|----------------|:----------------------:| +| Provider | Auto-Scaling Support | +|----------------|:--------------------:| | Playground | ❌ | | AWS | ✅ | | DigitalOcean | ❌ | diff --git a/docs/tutorials/migrating-from-heroku.md b/docs/tutorials/migrating-from-heroku.md index c80315b43..e091c110e 100644 --- a/docs/tutorials/migrating-from-heroku.md +++ b/docs/tutorials/migrating-from-heroku.md @@ -13,7 +13,7 @@ This tutorial will guide you through the process of migrating your Heroku applic * [The Heroku CLI (optional, but recommended)](https://devcenter.heroku.com/articles/heroku-cli#install-the-heroku-cli) * Cloud Account Credentials * [AWS](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-authentication.html) - * [GCP](https://cloud.google.com/docs/authentication/set-up-adc-local-dev-environment) (coming soon) + * [GCP](https://cloud.google.com/docs/authentication/set-up-adc-local-dev-environment) :::tip **Do I need a Dockerfile?** @@ -167,7 +167,7 @@ Now all you need to do is deploy your application to the cloud. ### Deploying to AWS -If you're deploying to AWS, you'll need to invoke `defang compose up` with your AWS access credentials in the environment: +If you're deploying to AWS, you'll need to invoke `defang compose up --provider aws` with your AWS access credentials in the environment:
@@ -285,7 +285,86 @@ See our full tutorial on [deploying to AWS](/docs/tutorials/deploy-to-aws). ### Deploying to GCP -GCP support for deployments without Dockerfiles coming soon. +If you're deploying to GCP, you'll need to invoke `defang compose up --provider gcp` with your GCP access credentials in the environment: + +
+ +Run the following command +``` +GCP_PROJECT_ID=my-project-123456 defang compose up --provider gcp +``` + +``` + * Using Google Cloud Platform provider from command line flag + ! Defang cannot monitor status of the following managed service(s): [postgres release]. + To check if the managed service is up, check the status of the service which depends on it. + ! service "postgres": missing memory reservation; using provider-specific defaults. Specify deploy.resources.reservations.memory to avoid out-of-memory errors + ! service "web": ingress port without healthcheck defaults to GET / HTTP/1.1 + * Packaging the project files for web at /Users/defang/wk/vast-badlands + * Uploading the project files for web + * Setting up defang CD in GCP project my-project-123456, this could take a few minutes + * Packaging the project files for release at /Users/defang/wk/vast-badlands + * Uploading the project files for release + * Tailing logs for deployment ID fgdh4ct0hopz ; press Ctrl+C to detach: + * Showing only build logs and runtime errors. Press V to toggle verbose mode. +2025-09-08T10:28:08.634-04:00 cd ** Update started for stack beta +2025-09-08T10:28:17.990-04:00 cd ** Updating service "postgres" +2025-09-08T10:28:44.773-04:00 cd ** Building image for "release"... +2025-09-08T10:28:58.012-04:00 cd ** Building image for "web"... +2025-09-08T10:47:02.479-04:00 release I, [2025-09-08T14:47:02.409371 #1] INFO -- : Migrating to CreateMembers (20240416182733) +2025-09-08T10:47:02.479-04:00 release == 20240416182733 CreateMembers: migrating ==================================== +2025-09-08T10:47:02.479-04:00 release -- create_table(:members) +2025-09-08T10:47:02.479-04:00 release -> 0.0073s +2025-09-08T10:47:02.479-04:00 release == 20240416182733 CreateMembers: migrated (0.0073s) =========================== +2025-09-08T10:47:02.479-04:00 release I, [2025-09-08T14:47:02.432075 #1] INFO -- : Migrating to DeviseCreateUsers (20240417165503) +2025-09-08T10:47:02.479-04:00 release == 20240417165503 DeviseCreateUsers: migrating ================================ +2025-09-08T10:47:02.479-04:00 release -- create_table(:users) +2025-09-08T10:47:02.479-04:00 release -> 0.0080s +2025-09-08T10:47:02.479-04:00 release -- add_index(:users, :email, {:unique=>true}) +2025-09-08T10:47:02.479-04:00 release -> 0.0022s +2025-09-08T10:47:02.479-04:00 release -- add_index(:users, :reset_password_token, {:unique=>true}) +2025-09-08T10:47:02.479-04:00 release -> 0.0021s +2025-09-08T10:47:02.479-04:00 release == 20240417165503 DeviseCreateUsers: migrated (0.0124s) ======================= +2025-09-08T10:47:02.479-04:00 release I, [2025-09-08T14:47:02.458370 #1] INFO -- : Migrating to AddUserIdToMembers (20240417202202) +2025-09-08T10:47:02.479-04:00 release == 20240417202202 AddUserIdToMembers: migrating =============================== +2025-09-08T10:47:02.479-04:00 release -- add_column(:members, :user_id, :integer) +2025-09-08T10:47:02.479-04:00 release -> 0.0044s +2025-09-08T10:47:02.479-04:00 release -- add_index(:members, :user_id) +2025-09-08T10:47:02.479-04:00 release -> 0.0019s +2025-09-08T10:47:02.479-04:00 release == 20240417202202 AddUserIdToMembers: migrated (0.0064s) ====================== +2025-09-08T10:48:16.157-04:00 web => Booting Puma +2025-09-08T10:48:16.157-04:00 web => Rails 7.1.3.2 application starting in production +2025-09-08T10:48:16.157-04:00 web => Run `bin/rails server --help` for more startup options +2025-09-08T10:48:18.625-04:00 web [1] Puma starting in cluster mode... +2025-09-08T10:48:18.625-04:00 web [1] * Puma version: 6.4.2 (ruby 3.3.4-p94) ("The Eagle of Durango") +2025-09-08T10:48:18.625-04:00 web [1] * Min threads: 5 +2025-09-08T10:48:18.625-04:00 web [1] * Max threads: 5 +2025-09-08T10:48:18.625-04:00 web [1] * Environment: production +2025-09-08T10:48:18.626-04:00 web [1] * Master PID: 1 +2025-09-08T10:48:18.626-04:00 web [1] * Workers: 2 +2025-09-08T10:48:18.626-04:00 web [1] * Restarts: (✔) hot (✔) phased +2025-09-08T10:48:18.626-04:00 web [1] * Listening on http://0.0.0.0:5000 +2025-09-08T10:48:18.627-04:00 web [1] Use Ctrl-C to stop +``` +
+ +### Deploying to DigitalOcean + +If you're deploying to GCP, you'll need to invoke `defang compose up --provider digitalocean` with your DigitalOcean access credentials in the environment: + +:::warning +Some Heroku applications may require some manual adjustments to be deployed to DigitalOcean with Defang. +Dockerfiles are required for deployments to DigitalOcean, so you may need to create one if your application does not already have one. Support for automatic Dockerfile generation is coming soon with [Railpack](/docs/concepts/railpack.md). +Defang also does not yet support [one-off jobs](/docs/concepts/one_off_jobs) on DigitalOcean, so you will need to modify your services to run these tasks during initialization. +::: + +``` +export DIGITALOCEAN_TOKEN=your_digitalocean_token +export SPACES_ACCESS_KEY_ID=your_spaces_access_key_id +export SPACES_SECRET_ACCESS_KEY=your_spaces_secret_access_key +$ defang compose up --provider digitalocean +``` + ## Step 3 - Migrating your data