Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 6 additions & 7 deletions docs/concepts/defang-byoc.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ $ export DEFANG_PROVIDER=digitalocean

Please read the [AWS Provider](../providers/aws/aws.md) documentation for more details about how the AWS provider works and how to get started.

:::success AWS Free Tier & Credits
You can use the AWS Free Tier to try out Defang. Learn more about it [here](https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all). If you're an elligible startup, you can sign up for credits [here](https://aws.amazon.com/startups/sign-up?referrer_url_path=%2Fstartups).
:::tip[AWS Free Tier & Credits]
You can use the AWS Free Tier to try out Defang. Learn more about it [here](https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all). If you're an eligible startup, you can sign up for credits [here](https://aws.amazon.com/startups/sign-up?referrer_url_path=%2Fstartups).
:::

## DigitalOcean
Expand All @@ -44,8 +44,8 @@ The Defang DigitalOcean Provider is available for Public Preview as of October 2

Please read the [DigitalOcean Provider](../providers/digitalocean/digitalocean.md) documentation for more details about how the DigitalOcean provider works and how to get started.

:::success DigitalOcean Credits
You can get DigitalOcean credits to try out Defang. Learn more about it on their [pricing page](https://www.digitalocean.com/pricing). If you're an elligible startup, you can sign up for credits [here](https://www.digitalocean.com/hatch).
:::tip[DigitalOcean Credits]
You can get DigitalOcean credits to try out Defang. Learn more about it on their [pricing page](https://www.digitalocean.com/pricing). If you're an eligible startup, you can sign up for credits [here](https://www.digitalocean.com/hatch).
:::

## GCP
Expand All @@ -56,11 +56,10 @@ The Defang GCP Provider is available for Public Preview as of December 2024.

Please check out the [GCP Provider](../providers/gcp/) page for more details.

:::success GCP Free Tier & Credits
You can use the GCP Free Tier to try out Defang. Learn more about it [here](https://cloud.google.com/free). If you're an elligible startup, you can sign up for credits [here](https://cloud.google.com/developers/startups).
:::tip[GCP Free Tier & Credits]
You can use the GCP Free Tier to try out Defang. Learn more about it [here](https://cloud.google.com/free). If you're an eligible startup, you can sign up for credits [here](https://cloud.google.com/developers/startups).
:::


## Azure

:::info
Expand Down
2 changes: 1 addition & 1 deletion docs/concepts/managed-storage/managed-object-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ sidebar_position: 3000

# Managed Object Storage

:::info Not Yet Supported
:::info[Not Yet Supported]
As of September 2024, Defang does not yet support managed Object Storage, but it is on our roadmap. If you are interested in Object Storage support, please vote on [this issue](https://github.com/DefangLabs/defang/issues/688).
:::

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ You can also set the following optional environment variables to configure the m
You can connect to the managed Postgres instance using the name of your service as the hostname, `POSTGRES_USER`, `POSTGRES_DB`, and `POSTGRES_PASSWORD` environment variables.

### Example

:::info
For a smoother experience with Defang, we recommend using Postgres 14 for your container images. This version provides easier access and improved usability.
:::
Expand Down Expand Up @@ -83,7 +84,7 @@ When a project is deployed to a [production environment](/docs/concepts/deployme

The AWS Console can be used to restore a snapshot into a new instance of Postgres.

<!--
{/*
### Major Version Updating of Engine

To update the database engine you can update the image to a later version in your Compose file and apply it via ```defang compose up --provider=aws```. In the example below, we change from Postgres 15 to 16.
Expand All @@ -100,4 +101,5 @@ to
```
database:
image: postgres:16
``` -->
```
*/}
3 changes: 1 addition & 2 deletions docs/concepts/managed-storage/managed-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,4 @@ sidebar_position: 000

# Managed Storage

Defang helps you provision the infrastructure you need to run your services. That infrastructure is designed to scale in and out without persistent storage, so you can build highly scalable services. But Defang can also help you provision managed services to store and persist your data, like [caches](./managed-redis.md), [databases](./managed-postgres.md), and [object storage](./managed-object-storage.md).

Defang helps you provision the infrastructure you need to run your services. That infrastructure is designed to scale in and out without persistent storage, so you can build highly scalable services. But Defang can also help you provision managed services to store and persist your data, like [caches](./managed-redis.md), [databases](./managed-postgres.mdx), and [object storage](./managed-object-storage.md).
20 changes: 10 additions & 10 deletions docs/concepts/resources.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ services:
replicas: 3
resources:
reservations:
cpus: '1.0'
cpus: "1.0"
memory: 2048M
devices:
- capabilities: ["gpu"]
Expand All @@ -32,17 +32,17 @@ const service = new defang.DefangService("gpu-service", {
deploy: {
replicas: 3,
resources: {
reservations: {
cpu: 1.0,
memory: 2048,
devices: [{capabilities: ['gpu']}]
}
}
}
reservations: {
cpu: 1.0,
memory: 2048,
devices: [{ capabilities: ["gpu"] }],
},
},
},
});
```

:::info GPUs
:::info[GPUs]
If you require access to GPUs, you can specify this in the `deploy.resources.reservations.devices[0].capabilities` section of your service as in the examples above. You can learn more about this in the [Docker-Compose documentation](https://docs.docker.com/compose/gpu-support/). This is the only supported value in the `deploy.resources.reservations.devices` section.
:::

Expand All @@ -68,7 +68,7 @@ Defang uses `shm_size` to configure both the memory and disk space available to
The default `shm_size` values for each platform are as follows. More or less may be specified.

| Platform | `shm_size` Minimum |
|---------------|--------------------|
| ------------- | ------------------ |
| AWS | 16G |
| Digital Ocean | 8G |
| GCP | 16G |
6 changes: 3 additions & 3 deletions docs/providers/aws/aws.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ sidebar_position: 000

Why should you use Defang with AWS? Defang allows you to easily create and manage full, scalable applications with AWS. Defang aims to make it easier to deploy your services to the cloud. Don't waste your time learning the ins and outs of AWS, deciding which of the 200+ services to use, and then writing the infrastructure code to deploy your services, and making sure they are properly secured. Defang does all of that for you.

:::success AWS Free Tier & Credits
You can use the AWS Free Tier to try out Defang. Learn more about it [here](https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all). If you're an elligible startup, you can sign up for credits [here](https://aws.amazon.com/startups/sign-up?referrer_url_path=%2Fstartups).
:::tip[AWS Free Tier & Credits]
You can use the AWS Free Tier to try out Defang. Learn more about it [here](https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all). If you're an eligible startup, you can sign up for credits [here](https://aws.amazon.com/startups/sign-up?referrer_url_path=%2Fstartups).
:::

## Getting Started
Expand Down Expand Up @@ -66,7 +66,7 @@ Defang can help you provision [managed storage](/docs/concepts/managed-storage/m

### Managed Postgres

When using [Managed Postgres](/docs/concepts/managed-storage/managed-postgres.md), the Defang CLI provisions an RDS Postgres instance in your account.
When using [Managed Postgres](/docs/concepts/managed-storage/managed-postgres.mdx), the Defang CLI provisions an RDS Postgres instance in your account.

### Managed Redis

Expand Down
11 changes: 8 additions & 3 deletions docs/providers/digitalocean/digitalocean.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ sidebar_position: 010
The Defang DigitalOcean Provider is available for Public Preview as of October 2024.
:::

:::success DigitalOcean Credits
You can get DigitalOcean credits to try out Defang. Learn more about it on their [pricing page](https://www.digitalocean.com/pricing). If you're an elligible startup, you can sign up for credits [here](https://www.digitalocean.com/hatch).
:::tip[DigitalOcean Credits]
You can get DigitalOcean credits to try out Defang. Learn more about it on their [pricing page](https://www.digitalocean.com/pricing). If you're an eligible startup, you can sign up for credits [here](https://www.digitalocean.com/hatch).
:::

Why should you use Defang with DigitalOcean? Defang allows you to easily create and manage full, scalable applications with DigitalOcean. Defang aims to make it easier to deploy your services to the cloud. DigitalOcean is one of the most popular cloud providers in the world and with Defang, you can bypass the complexities of the DigitalOcean platform. Let Defang do it for you and spend more time working on what's important to you!
Expand All @@ -23,12 +23,15 @@ Why should you use Defang with DigitalOcean? Defang allows you to easily create
To get started with the Defang BYOC DigitalOcean Provider, first [install the latest version of the Defang CLI](../getting-started#authenticate-with-defang).

### Sign up for DigitalOcean

Next, make sure you have signed up for a [DigitalOcean account](https://try.digitalocean.com/freetrialoffer/).

### Authenticate with DigitalOcean

After signing up for your account, be sure to set up your [personal access token](https://docs.digitalocean.com/reference/api/create-personal-access-token/). Defang will need to find this value in your shell as the `DIGITALOCEAN_TOKEN` environment variable.

### Authenticate with DigitalOcean Spaces

You will also need a [DigitalOcean Spaces access key](https://docs.digitalocean.com/products/spaces/how-to/manage-access/). Defang will need to find this value in your shell as the `SPACES_ACCESS_KEY_ID`, and `SPACES_SECRET_ACCESS_KEY` environment variables.

### Configure your shell environment
Expand All @@ -44,6 +47,7 @@ The Defang CLI will automatically check if these envinonment variables are set b
### Deploy your project to DigitalOcean

Once you are ready to go, add the `--provider=digitalocean` to your command to tell the Defang CLI to use the DigitalOcean provider or set the `DEFANG_PROVIDER` environment variable to `digitalocean`.

```bash
$ defang compose up --provider=digitalocean
# or
Expand Down Expand Up @@ -73,9 +77,10 @@ Defang allows you to configure your services with [sensitive config values](http
### Future Improvements

The following features are still in development for DigitalOcean:

- [Custom Domains](/docs/concepts//domains.mdx)
- [Managed Redis](/docs/concepts//managed-storage/managed-redis.md)
- [Managed Postgres](/docs/concepts/managed-storage/managed-postgres.md)
- [Managed Postgres](/docs/concepts/managed-storage/managed-postgres.mdx)
- [Managed Language Models](/docs/concepts/managed-llms/managed-language-models.md)

Stay tuned for future updates!
7 changes: 4 additions & 3 deletions docs/providers/gcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ sidebar_position: 3000
The Defang GCP Provider is available for Public Preview as of December 2024.
:::

:::success GCP Free Tier & Credits
You can use the GCP Free Tier to try out Defang. Learn more about it [here](https://cloud.google.com/free). If you're an elligible startup, you can sign up for credits [here](https://cloud.google.com/developers/startups).
:::tip[GCP Free Tier & Credits]
You can use the GCP Free Tier to try out Defang. Learn more about it [here](https://cloud.google.com/free). If you're an eligible startup, you can sign up for credits [here](https://cloud.google.com/developers/startups).
:::

Defang enables you to effortlessly develop and deploy full, scalable applications with GCP. It is designed to simplify deploying your services to the cloud. As one of the leading cloud providers globally, GCP offers powerful tools and resources, and with Defang, you can bypass the complexities of the GCP platform. Let Defang handle the heavy lifting so you can focus on what matters most to you!
Expand Down Expand Up @@ -66,10 +66,11 @@ Defang offers integration with managed, cloud-native large language model servic
### Future Improvements

The following features are in active development for GCP:

- [Configuration and management of secrets](/docs/concepts//configuration.md)
- [Networking and Load Balancing](/docs/concepts//networking.mdx)
- [Custom Domains](/docs/concepts//domains.mdx)
- [Managed Redis](/docs/concepts//managed-storage/managed-redis.md)
- [Managed Postgres](/docs/concepts/managed-storage/managed-postgres.md)
- [Managed Postgres](/docs/concepts/managed-storage/managed-postgres.mdx)

Stayed tuned for future updates!
86 changes: 48 additions & 38 deletions docs/tutorials/deploy-with-gpu.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,27 +5,36 @@ sidebar_position: 500

# Deploy a GPU-Powered Application to AWS

This tutorial will show you how to create and deploy a GPU-powered application on AWS using Defang.
This tutorial will show you how to create and deploy a GPU-powered application on AWS using Defang.

We will walk you through the whole deployment process based on the [Mistral & vLLM](https://github.com/DefangLabs/samples/tree/main/samples/vllm) sample.

:::info
Note that GPU deployments are not permitted on the Defang Playground. You must [upgrade to a paid account](https://defang.io/pricing/) and [deploy to your own cloud account](https://docs.defang.io/docs/tutorials/deploy-to-your-cloud).
:::

<iframe width="560" height="315" src="https://www.youtube.com/embed/kynFa2zU7hQ?si=qdV0xa6vkhMFJ6qv" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
<iframe
width="560"
height="315"
src="https://www.youtube.com/embed/kynFa2zU7hQ?si=qdV0xa6vkhMFJ6qv"
title="YouTube video player"
frameborder="0"
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
referrerpolicy="strict-origin-when-cross-origin"
allowfullscreen
></iframe>

## Prerequisites

* [A Defang Account](/docs/concepts/authentication)
* [The Defang CLI](/docs/getting-started#install-the-defang-cli)
* [AWS Account Credentials](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-authentication.html)
- [A Defang Account](/docs/concepts/authentication)
- [The Defang CLI](/docs/getting-started#install-the-defang-cli)
- [AWS Account Credentials](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-authentication.html)

### AWS Account with GPU Access

For any of this to work, you'll need to have access to GPU instances in your AWS account. To do that you'll need to go to the "[Service Quotas](https://console.aws.amazon.com/servicequotas/home)" console in your AWS account. From there you can request access to spot GPU instances. You'll need to request 8 or more because the value is per vCPU and the smallest GPU instance has 8 vCPUs. The instance types you're requesting are "All G and VT spot instances".

:::warning Timing
:::warning[Timing]
This process can take a few days for AWS to approve.
:::

Expand All @@ -41,27 +50,29 @@ You'll need to clone the [Mistral & vLLM](https://github.com/DefangLabs/samples/

## Step 2 - Check your [Defang BYOC](../concepts/defang-byoc.md) settings

* Make sure you [install the latest version of the Defang CLI](../getting-started#install-the-defang-cli.md)
* Then, make sure you have properly [authenticated your AWS account](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html). The Defang CLI makes use of AWS environment variables like `AWS_PROFILE`, `AWS_REGION`, `AWS_ACCESS_KEY_ID`, and `AWS_SECRET_ACCESS_KEY`, so make sure the correct values are set for those.
- Make sure you [install the latest version of the Defang CLI](../getting-started#install-the-defang-cli.md)
- Then, make sure you have properly [authenticated your AWS account](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html). The Defang CLI makes use of AWS environment variables like `AWS_PROFILE`, `AWS_REGION`, `AWS_ACCESS_KEY_ID`, and `AWS_SECRET_ACCESS_KEY`, so make sure the correct values are set for those.

:::tip
If you have the AWS CLI installed (which is not required in order to use the Defang CLI), you can verify that you've authenticated to AWS by running `aws sts get-caller-identity` and see your account ID.
:::

## Step 3 - Prepare your Environment
* Log into your Defang account

- Log into your Defang account

```bash
defang login
```

* Set the HuggingFace token using the `defang config` command
- Set the HuggingFace token using the `defang config` command

```bash
defang config set --name HF_TOKEN
```

[Configuration](../concepts/configuration.md) stores your sensitive information such as API keys, passwords, and other credentials for you.


## Step 4 - Explore the Compose File

The `compose.yml` file is where you define your services and their configurations.
Expand Down Expand Up @@ -105,19 +116,19 @@ We start with the latest vLLM Docker image provided by [Mistral AI](https://docs

```yaml
mistral:
image: ghcr.io/mistralai/mistral-src/vllm:latest
image: ghcr.io/mistralai/mistral-src/vllm:latest
```

We specify that we require a GPU to run our application.

```yaml
deploy:
resources:
reservations:
resources:
reservations:
cpus: '2.0'
memory: 8192M
devices:
- capabilities: ["gpu"]
memory: 8192M
devices:
- capabilities: ["gpu"]
```

The Mistral model will be downloaded from HuggingFace. We need to have a HuggingFace Token to enable the installation, so we specify that we need to get the `HF_TOKEN` configuration value from Defang.
Expand All @@ -126,41 +137,40 @@ Specifying the `HF_TOKEN` in the `environment` section of the service in the `co

```yaml
environment:
- HF_TOKEN
- HF_TOKEN
```

### The UI Service

In this sample we also provide a simple UI to interact with the endpoint created by vLLM. The UI service is a Next.js application that runs on port 3000.

:::tip Networking
:::tip
You can see here how Defang's [networking](../concepts//networking.mdx) works. The `mistral` service is available at `http://mistral:8000`, exactly as it would be in a local `docker-compose` environment.
:::

```yaml
ui:
restart: unless-stopped
build:
context: ui
dockerfile: Dockerfile
ports:
- mode: ingress
target: 3000
deploy:
resources:
reservations:
memory: 256M
healthcheck:
test: ["CMD","wget","--spider","http://localhost:3000"]
interval: 10s
timeout: 2s
retries: 10
ui:
restart: unless-stopped
build:
context: ui
dockerfile: Dockerfile
ports:
- mode: ingress
target: 3000
deploy:
resources:
reservations:
memory: 256M
healthcheck:
test: ["CMD", "wget", "--spider", "http://localhost:3000"]
interval: 10s
timeout: 2s
retries: 10
environment:
// highlight-next-line
- OPENAI_BASE_URL=http://mistral:8000/v1/
- OPENAI_BASE_URL=http://mistral:8000/v1/
```


## Step 5 - Deploy to Your Own AWS Account with Defang

Run the following command to deploy your service:
Expand Down
8 changes: 8 additions & 0 deletions docusaurus.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ const config = {
'https://github.com/DefangLabs/defang-docs/tree/main/',
},
blog: {
blogSidebarCount: 'ALL', // Show all posts in sidebar
blogSidebarTitle: 'All Posts', // Custom title
showReadingTime: true,
// Remove the editUrl property for the blog section
},
Expand Down Expand Up @@ -188,6 +190,12 @@ const config = {
],
markdown: {
mermaid: true,
format: 'mdx',
mdx1Compat: { //set to false for Docusaurus v3+ compatibility
comments: false,
admonitions: false,
headingIds: false
}
},
};

Expand Down