diff --git a/blog/2024-03-28-slackbot-sample.md b/blog/2024-03-28-slackbot-sample.md index 629fff268..035f8208f 100644 --- a/blog/2024-03-28-slackbot-sample.md +++ b/blog/2024-03-28-slackbot-sample.md @@ -13,7 +13,7 @@ Before we dive into the details, let's make sure you have everything you need to 1. **Install Defang CLI:** Simplify your deployment process by installing the Defang CLI tool. Follow the instructions [here](https://docs.defang.io/docs/getting-started/installing) to get it up and running quickly. -2. **Slack API Token:** Create a Slack App at https://api.slack.com/apps, granting it the necessary permissions, including the bot `chat:write` scope. +2. **Slack API Token:** Create a Slack App at https://api.slack.com/apps, granting it the necessary permissions, including the bot `chat:write` scope. ![screenshot of the slack admin UI showing the bot scopes](/img/slackbot-sample/scopes.png) 3. **Install the app in your workspace:** You'll need to install the app in your workspace for it to work. Click the "Install to Workspace" button in the Slack admin UI to do this. Mine says "Reinstall" because I've already installed it. @@ -36,12 +36,12 @@ cd defang/samples/golang/slackbot Now that we have everything set up, let's dive into the deployment process. Follow these steps to deploy your Slackbot effortlessly: -1. **Set Up Secrets:** Prioritize security by configuring environment variables as secrets. Use the Defang CLI's `defang secret set` command to set the `SLACK_TOKEN` and `SLACK_CHANNEL_ID` secrets. +1. **Set Up Secrets:** Prioritize security by configuring environment variables as secrets. Use the Defang CLI's `defang config set` command to set the `SLACK_TOKEN` and `SLACK_CHANNEL_ID` secrets. Replace `your_slack_token` and `your_slack_channel_id` with the respective values: ```bash - defang secret set --name SLACK_TOKEN --value your_slack_token - defang secret set --name SLACK_CHANNEL_ID --value your_slack_channel_id + defang config set --name SLACK_TOKEN --value your_slack_token + defang config set --name SLACK_CHANNEL_ID --value your_slack_channel_id ``` 2. **Deploy the Slackbot:** Use the Defang CLI's `defang compose up` command to deploy. diff --git a/docs/concepts/compose.md b/docs/concepts/compose.md index 7efbc25a3..c8acbd9c3 100644 --- a/docs/concepts/compose.md +++ b/docs/concepts/compose.md @@ -20,10 +20,10 @@ One thing to keep in mind is that, at the time of this writing, Defang identifie ## Configuration -If you have a service that depends on a secret like an api key, you can set that [secret](./secrets.md) using the CLI: +If you have a service that depends on a secret like an api key, you can set that [secret](./configuration.md) using the CLI: ``` -defang secret set --name MY_API_KEY +defang config set --name MY_API_KEY ``` and then connect it to the service by specifying it in the `compose.yaml`: @@ -40,5 +40,5 @@ secrets: ``` :::info Configuration & Secrets -Read more about configuration in the [configuration page](./configuration.md) and about secrets in the [secrets page](./secrets.md). +Read more about configuration in the [configuration page](./configuration.md) and about secrets in the [secrets page](./configuration.md). ::: \ No newline at end of file diff --git a/docs/concepts/configuration.md b/docs/concepts/configuration.md index d3e36041d..d60f6b26a 100644 --- a/docs/concepts/configuration.md +++ b/docs/concepts/configuration.md @@ -1,6 +1,6 @@ --- title: Configuration -description: Configuring your Defang application. +description: Configuring your Defang application, including sensitive config values like API keys, passwords, and other credentials. sidebar_position: 225 --- @@ -8,8 +8,36 @@ sidebar_position: 225 Defang allows you to configure your application using environment variables. You can set environment variables in your [`compose.yaml` file](./compose.md), or in your [Pulumi program](./pulumi.md). Using Pulumi gives you the advantage of being able to manage your environment variables across different environments using Pulumi stacks. -You can also use [secrets](./secrets.md) to store sensitive information like API keys and database passwords. - :::tip Sample You can find a sample of how to set environment variables with Pulumi [here](https://github.com/defang-io/defang/tree/main/samples/nodejs/remix-aiven-postgres). -::: \ No newline at end of file +::: + +# Sensitive Config aka Secrets + +The Defang CLI allows you to securely store sensitive information such as API keys, passwords, and other credentials. + +You can use sensitive config by specifying them in the `environment` section of a service in a `compose.yaml` file without any value, or by specifying an environment key with a `null` value in your Pulumi code. + +```ts +services: + service1: + image: image1:latest + environment: + - API_KEY +``` + +Use the `defang config` command of the Defang CLI to manage the values. + +## Connecting Services + +If you have created a service before a secret you can connect it by running the `defang compose start` command if using the [`defang compose` workflow](./compose.md). If you are using the [Pulumi-based workflow](./pulumi.md) you will need to redeploy using Pulumi. + +:::tip Sample +You can find a sample of how to set sensitive config values [here](https://github.com/defang-io/defang/tree/main/samples/nodejs/ChatGPT%20API). +::: + +## Providers + +Here are the different ways sensitive config values are stored depending on the provider you are using: + + * [AWS](../providers/aws.md#secrets) diff --git a/docs/concepts/secrets.md b/docs/concepts/secrets.md deleted file mode 100644 index b4d2c54e2..000000000 --- a/docs/concepts/secrets.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -title: Secrets -description: Secrets are a way to securely store sensitive information such as API keys, passwords, and other credentials. -sidebar_position: 250 ---- - -# Secrets - -The Defang CLI allows you to securely store sensitive information such as API keys, passwords, and other credentials. - -You can use secrets by specifying them in the `secrets` section of a service in a `compose.yaml` file, or by specifying a secret in your Pulumi code. - -## Connecting Services - -If you have created a service before a secret you can connect it by running the `defang compose start` command if using the [`defang compose` workflow](./compose.md). If you are using the [Pulumi-based workflow](./pulumi.md) you will need to redeploy using Pulumi. - -:::tip Sample -You can find a sample of how to set secrets [here](https://github.com/defang-io/defang/tree/main/samples/nodejs/ChatGPT%20API). -::: - -## Providers - -Here are the different ways secrets are stored depending on the provider you are using: - - * [AWS](../providers/aws.md#secrets) diff --git a/docs/faq.md b/docs/faq.md index b0d67e3a0..36321b09c 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -132,7 +132,7 @@ secrets: ``` ### "unsupported secret …: not marked external:true" -- This message is displayed when you run `defang compose up` and the Compose file declares a `secret` that is not marked `external:true`. Defang only supports external secrets, managed by the `defang secret` command. To silence the warning, mark the secret as `external:true` in the top-level `secrets` section: +- This message is displayed when you run `defang compose up` and the Compose file declares a `secret` that is not marked `external:true`. Defang only supports external secrets, managed by the `defang config` command. To silence the warning, mark the secret as `external:true` in the top-level `secrets` section: ``` … secrets: diff --git a/docs/intro.md b/docs/intro.md index 3b52bc653..694ce1b56 100644 --- a/docs/intro.md +++ b/docs/intro.md @@ -25,7 +25,7 @@ Defang provides a streamlined experience to develop, deploy, observe, and update - Automated [Dockerfile builds](./concepts/deployments.md) - Support for [pre-built Docker containers](./tutorials/deploy-container-using-the-cli.mdx), from public or private image registries - Ability to express your project configuration using a [Docker Compose YAML](./concepts/compose.md) file -- Ability to manage encrypted [secrets](./concepts/secrets.md) and [configuration](./concepts/configuration.md) +- Ability to manage [encrypted configuration values](./concepts/configuration.md) - Pre-configured environments with built-in [security](./concepts/security.md), [networking](./concepts/networking.mdx), and [observability](./concepts/observability.md) - [One-command deployments](./getting-started/installing.md) - Support for [GPUs](./concepts/resources.md) diff --git a/docs/providers/aws.md b/docs/providers/aws.md index f13424dc0..8049c21ff 100644 --- a/docs/providers/aws.md +++ b/docs/providers/aws.md @@ -21,7 +21,7 @@ If you have the aws CLI installed, you should be able to successfully run `aws s ::: :::warning -The Defang CLI does not depend on the AWS CLI. It uses the [AWS SDK for Go](https://aws.amazon.com/sdk-for-go/) to interact with your AWS account. In most cases, if you can run the `aws sts get-caller-identity` from the tip above, you should be good to go. However, due to a difference between the AWS CLI and the AWS SDK for Go, there is at least one case where they behave differently: if you are using `aws sso login` and have clashing profiles in your `.aws/config` and `.aws/credentials` files, the AWS CLI will prioritize SSO profiles and caches over regular profiles, but the AWS SDK for Go will prioritize the credentials file, and it may fail. +The Defang CLI does not depend on the AWS CLI. It uses the [AWS SDK for Go](https://aws.amazon.com/sdk-for-go/) to interact with your AWS account. In most cases, if you can run the `aws sts get-caller-identity` from the tip above, you should be good to go. However, due to a difference between the AWS CLI and the AWS SDK for Go, there is at least one case where they behave differently: if you are using `aws sso login` and have clashing profiles in your `.aws/config` and `.aws/credentials` files, the AWS CLI will prioritize SSO profiles and caches over regular profiles, but the AWS SDK for Go will prioritize the credentials file, and it may fail. ::: ## Region @@ -34,7 +34,7 @@ Defang uses resources that are native to the cloud provider you are using. The f ### Secrets -Defang allows you to configure your services with secrets. Secrets are stored in AWS Systems Manager Parameter Store, and are encrypted. +Defang allows you to configure your services with sensitive config values. Sensitive values are stored in AWS Systems Manager Parameter Store, and are encrypted. ### Deployment diff --git a/docs/samples.md b/docs/samples.md index 1779f7be9..6c59218c8 100644 --- a/docs/samples.md +++ b/docs/samples.md @@ -12,4 +12,160 @@ Check out our sample projects here to get some inspiration and get a sense of ho import Samples from "../src/components/Samples"; - \ No newline at end of file +Dockerfile to describe the basic image of your applications.\n2. A docker-compose file to define and run multi-container Docker applications.\n3. A .dockerignore file to comply with the size limit (10MB).\n\n## Prerequisite\n1. Download Defang CLI\n2. If you are using Defang BYOC, make sure you have properly authenticated your AWS account\nPlus, make sure that you have properly set your environment variables like `AWS_PROFILE`, `AWS_REGION`, `AWS_ACCESS_KEY_ID`, and `AWS_SECRET_ACCESS_KEY`.\n\n## A Step-by-Step Guide\n1. Edit your content in the `_posts` directory\n2. Open the terminal and type `defang login`\n3. Type `defang compose up` in the CLI\n4. Now your application will be launched\n" + }, + { + "name": "chat", + "category": "nodejs", + "readme": "# Minimal Chat App\n\nThis is a minimal chat application that shows how to use Socket.IO with Node.js with minimal code deployed with Defang.\n\n## Prerequisites\n\nInstall the Defang CLI by following the instructions in the [Defang CLI documentation](https://docs.defang.io/docs/getting-started).\n\n## Build and run the application\n\nIf you have environment variables configured for your [own cloud account](https://docs.defang.io/docs/concepts/defang-byoc), this will deploy the application to your cloud account, otherwise it will deploy to the Defang cloud.\n\n```sh\ndefang compose up\n```" + }, + { + "name": "document-starter", + "category": "nodejs", + "readme": "This template is a documentation starter project developed using Nextra, designed to streamline the creation of your documentation and quickly build a digital knowledgebase. You can add content easily by simply adding markdown files. This code-free solution requires no adjustments to the basic structure. We have prepared all the essential files for deployment. By spending just a few minutes setting up the environment, as detailed in the prerequisites, and executing the commands in our step-by-step guide, your website will be ready to go live in no time!\n\n## Essential Setup Files\n1. A Dockerfile to describe the basic image of your applications.\n2. A docker-compose file to define and run multi-container Docker applications.\n3. A .dockerignore file to comply with the size limit (10MB).\n\n## Prerequisite\n1. Download Defang CLI\n2. If you are using Defang BYOC, make sure you have properly authenticated your AWS account\nPlus, make sure that you have properly set your environment variables like `AWS_PROFILE`, `AWS_REGION`, `AWS_ACCESS_KEY_ID`, and `AWS_SECRET_ACCESS_KEY`.\n\n## A Step-by-Step Guide\n1. Open the terminal and type `defang login`\n2. Type `defang compose up` in the CLI\n3. Now your application will be launched\n" + }, + { + "name": "minimal", + "category": "nodejs", + "readme": "# Minimal Node.js example\n\nThis example shows how to build a minimal Node.js application using the [Node.js](https://nodejs.org/) runtime.\n\n## Prerequisites\n\nInstall the Defang CLI by following the instructions in the [Defang CLI documentation](https://docs.defang.io/docs/getting-started).\n\n## Build and run the application\n\nIf you have environment variables configured for your [own cloud account](https://docs.defang.io/docs/concepts/defang-byoc), this will deploy the application to your cloud account, otherwise it will deploy to the Defang cloud.\n\n```sh\ndefang compose up\n```" + }, + { + "name": "nextjs", + "category": "nodejs", + "readme": "# Next.js × Defang\n\nA basic Next.js app with a Dockerfile and docker-compose ready to deploy to AWS with [Defang](https://defang.io).\n\n## Steps\n\n1. [Install Defang](https://docs.defang.io/docs/getting-started/installing)\n2. [Authenticate with Defang](https://docs.defang.io/docs/getting-started/authenticating)\n3. (optional) [Authenticate with AWS](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html)\n4. Run `defang compose up` in the root of this project" + }, + { + "name": "nextjs-boilerplate", + "category": "nodejs", + "readme": "This template is a boilerplate project developed using Next.js, offering a starting point to help you quickly build your website. We have prepared all the necessary files for deployment. By spending just a few minutes setting up the environment, as detailed in the prerequisites, and executing the commands in our step-by-step guide, your website will be ready to go live in no time!\n\n## Essential Setup Files\n1. A Dockerfile to describe the basic image of your applications.\n2. A docker-compose file to define and run multi-container Docker applications.\n3. A .dockerignore file to comply with the size limit (10MB).\n\n## Prerequisite\n1. Download Defang CLI\n2. If you are using Defang BYOC, make sure you have properly authenticated your AWS account\nPlus, make sure that you have properly set your environment variables like `AWS_PROFILE`, `AWS_REGION`, `AWS_ACCESS_KEY_ID`, and `AWS_SECRET_ACCESS_KEY`.\n\n## A Step-by-Step Guide\n1. Open the terminal and type `defang login`\n2. Type `defang compose up` in the CLI\n3. Now your application will be launched\n" + }, + { + "name": "nextjs-github-actions", + "category": "nodejs", + "readme": "# Next.js × Defang\n\nA basic Next.js app with a Dockerfile and docker-compose ready to deploy to AWS with [Defang](https://defang.io).\n\n## Steps\n\n1. [Install Defang](https://docs.defang.io/docs/getting-started/installing)\n2. [Authenticate with Defang](https://docs.defang.io/docs/getting-started/authenticating)\n3. (optional) [Authenticate with AWS](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html)\n4. Run `defang compose up` in the root of this project" + }, + { + "name": "remix-aiven-postgres", + "category": "nodejs", + "readme": "# Full Stack Defang + Pulumi Example\n\nIn this example, we run a Remix application connected to a Postgres database using Prisma as an ORM. When we deploy our service to [Defang](https://defang.io/), we also deploy a Postgres service and database using [Aiven](https://aiven.io/) so we can run our full application in the cloud.\n\n## Running Locally\n\n\nTo run this example locally, you'll need to have a Postgres database. You can run one locally with Docker:\n\n```\ndocker run -p 5432:5432 -e POSTGRES_PASSWORD=password -d postgres\n```\n\nCreate a `.env` file with the following:\n\n```\nDATABASE_URL=\"postgresql://postgres:password@localhost:5432/postgres?schema=public\"\n```\n\nThen run `npm install` and `npm run dev` in the `remix` directory to start the application.\n\n\n## Deploying to Defang\n\nFirst, `cd` into the `pulumi` directory and make sure you're logged into Defang with `defang login` and into Pulumi with `pulumi login`.\n\nNext, head to your Aiven account and create an api token, then run the following command to store it in your Pulumi stack config:\n\n```\npulumi config set --secret aiven:apiToken \n```\n\nYou'll also need to make sure you have an Aiven organization with a billing method attached to a billing group. Get the organization id and the billing group id and add them to your config with the following commands:\n\n```\npulumi config set --secret aivenOrganizationId \npulumi config set --secret aivenBillingGroupId \n```\n\nNow, run `pulumi up` to deploy your application to Defang and Aiven! Head to the [portal](https://portal.defang.dev) to check on status, or run `defang services`.\n\n" + }, + { + "name": "imgproxy", + "category": "other", + "readme": "# ImgProxy\n\nImgProxy is a fast and secure standalone server for resizing and converting remote images. It's can be deployed using their official Docker image, as documented [here](https://docs.imgproxy.net/installation#docker).\n\n## Prerequisites\n\nInstall the Defang CLI by following the instructions in the [Defang CLI documentation](https://docs.defang.io/docs/getting-started).\n\n## Build and run the application\n\nIf you have environment variables configured for your [own cloud account](https://docs.defang.io/docs/concepts/defang-byoc), this will deploy the application to your cloud account, otherwise it will deploy to the Defang cloud.\n\n```sh\ndefang compose up\n```\n" + }, + { + "name": "pulumi", + "category": "other", + "readme": "# pulumi" + }, + { + "name": "vllm", + "category": "other", + "readme": "\n# Deploying Mistral with vLLM\n\nThis guide demonstrates how to deploy Mistral using VLM. You'll need a Hugging Face token to begin.\n\n## Prerequisites\n\n- Hugging Face token\n\n## Steps\n\n1. **Set the Hugging Face Token**\n\n First, set the Hugging Face token using the `defang config` command.\n\n ```bash\n defang config set --name HF_TOKEN\n ```\n\n2. **Launch with Defang Compose**\n\n Run the following command to start the services:\n\n ```bash\n defang compose up\n ```\n\n The provided `docker-compose.yml` file includes the Mistral service. It's configured to run on an AWS instance with GPU support. The file also includes a UI service built with Next.js, utilizing Vercel's AI SDK.\n\n > **OpenAI SDK:** We use the OpenAI sdk, but set the `baseURL` to our Mistral endpoint.\n\n > **Note:** The API route does not use a system prompt, as the Mistral model we're deploying currently does not support this feature. To get around this we inject a couple messages at the front of the conversation providing the context (see the `ui/src/app/api/chat/route.ts` file). Other than that, the integration with the OpenAI SDK should be structured as expected.\n\n > **Changing the content:** The content for the bot is set in the `ui/src/app/api/chat/route.ts` file. You can edit the prompt in there to change the behaviour. You'll notice that it also pulls from `ui/src/app/docs.md` to provide content for the bot to use. You can edit this file to change its \"knowledge\".\n\n## Configuration\n\n- The Docker Compose file is ready to deploy Mistral and the UI service.\n- The UI uses Next.js and Vercel's AI SDK for seamless integration.\n\nBy following these steps, you should be able to deploy Mistral along with a custom UI on AWS, using GPU capabilities for enhanced performance.\n" + }, + { + "name": "AI-ML Music", + "category": "python", + "readme": "# AI-ML Music" + }, + { + "name": "AWS S3", + "category": "python", + "readme": "\n## Setup\nThis sample requires an API key to access AWS S3. The name of the config values is referenced in the docker-compose.yml file.\nTo provide a value for it, you can use the Defang CLI like this:\n\n```\ndefang config set --name AWS_ACCESS_KEY\ndefang config set --name AWS_SECRET_KEY\n```\n\nand then enter the value when prompted.\n\n## Testing\ncurl -X POST -H 'Content-Type: application/json' -d '{ \"first_name\" : \"jane\", \"last_name\" : \"doe\" }' https://xxxxxx/upload\ncurl https://xxxxxx/download\n" + }, + { + "name": "Basic Service", + "category": "python", + "readme": "# Basic Service" + }, + { + "name": "ChatGPT API", + "category": "python", + "readme": "\n## Setup\nThis sample requires an API key to access the OpenAI API. The name of the config value is referenced in the docker-compose.yml file. To provide a value for it, you can use the Defang CLI like this:\n\n```\ndefang config set --name OPENAI_KEY\n```\n\nand then enter the value when prompted.\n\n\n## Testing\n```\necho \"Hello\" | curl -H \"Content-Type: application/text\" -d @- https://xxxxxxxx/prompt\n```\nor\n```\ncat prompt.txt | curl -H \"Content-Type: application/text\" -d @- https://xxxxxxxx/prompt\n```" + }, + { + "name": "Fetch and Return JSON", + "category": "python", + "readme": "# Fetch and Return JSON" + }, + { + "name": "Web Page and Form Post", + "category": "python", + "readme": "# Web Page and Form Post" + }, + { + "name": "django", + "category": "python", + "readme": "This is a simple example of how to run Django on Defang. It is a simple Todo app that uses SQLite as the database (so data is *not* persisted between deployments). We will be putting together an example with a managed database soon.\n\nThe app includes a management command which is run on startup to create a superuser with the username `admin` and password `admin`. This means you can login to the admin interface at `/admin/` and see the Django admin interface without any additional steps. The `example_app` is already registered and the `Todo` model is already set up to be managed in the admin interface.\n\nThe Dockerfile and compose files are already set up for you and are ready to be deployed. Serving is done using [Gunicorn](https://gunicorn.org/) and uses [WhiteNoise](https://whitenoise.readthedocs.io/en/latest/) for static files. The `CSRF_TRUSTED_ORIGINS` setting is configured to allow the app to run on a `defang.dev` subdomain.\n\n## Essential Setup Files\n1. A Dockerfile to describe the basic image of your applications.\n2. A docker-compose file to define and run multi-container Docker applications.\n3. A .dockerignore file to comply with the size limit (10MB).\n\n## Prerequisite\n1. Download Defang CLI\n2. If you are using Defang BYOC, make sure you have properly authenticated your AWS account\nPlus, make sure that you have properly set your environment variables like `AWS_PROFILE`, `AWS_REGION`, `AWS_ACCESS_KEY_ID`, and `AWS_SECRET_ACCESS_KEY`.\n\n## A Step-by-Step Guide\n1. (optional) If you are using Defang BYOC, make sure to update the `CSRF_TRUSTED_ORIGINS` setting in the `settings.py` file to include an appropriate domain.\n2. Open the terminal and type `defang login`\n3. Type `defang compose up` in the CLI\n4. Now your application will be launched\n" + }, + { + "name": "flask", + "category": "python", + "readme": "# flask" + }, + { + "name": "rails", + "category": "ruby", + "readme": "\nThis template is a member list project developed using Ruby on Rails, offering a starting point to help you quickly build your team management system. We have prepared all the essential files for deployment. By spending less than 10 minutes setting up the environment, as detailed in the prerequisites, and executing the commands in our step-by-step guide, your website will be ready to go live to the world!\n\n## Essential Setup Files\n1. A Dockerfile to describe the basic image of your applications.\n2. A docker-compose file to define and run multi-container Docker applications.\n3. A .dockerignore file to comply with the size limit (10MB).\n\n## Prerequisite\n1. Download Defang CLI\n2. If you are using Defang BYOC, make sure you have properly authenticated your AWS account\nPlus, make sure that you have properly set your environment variables like `AWS_PROFILE`, `AWS_REGION`, `AWS_ACCESS_KEY_ID`, and `AWS_SECRET_ACCESS_KEY`.\n\n## A Step-by-Step Guide\n1. Open the terminal and type `defang login`\n2. Type `defang compose up` in the CLI\n3. Now your application will be launched\n" + } +]} /> \ No newline at end of file diff --git a/docs/tutorials/deploy-with-gpu.mdx b/docs/tutorials/deploy-with-gpu.mdx index 3d9a07bf9..d54f5f7ec 100644 --- a/docs/tutorials/deploy-with-gpu.mdx +++ b/docs/tutorials/deploy-with-gpu.mdx @@ -13,7 +13,7 @@ This tutorial guides you to create and deploy a GPU-powered application on AWS u ### AWS Account with GPU Access -For any of this to work, you'll need to have access to GPU instances in your AWS account. To do that you'll need to go to the "[Service Quotas](https://console.aws.amazon.com/servicequotas/home)" console in your AWS account. From there you can request access to spot GPU instances. You'll need to request 8 or more because the value is per vCPU and the smallest GPU instance has 8 vCPUs. The instance types you're requesting are "All G and VT spot instances". +For any of this to work, you'll need to have access to GPU instances in your AWS account. To do that you'll need to go to the "[Service Quotas](https://console.aws.amazon.com/servicequotas/home)" console in your AWS account. From there you can request access to spot GPU instances. You'll need to request 8 or more because the value is per vCPU and the smallest GPU instance has 8 vCPUs. The instance types you're requesting are "All G and VT spot instances". :::warning Timing This process can take a few days for AWS to approve. @@ -46,17 +46,17 @@ aws sts get-caller-identity defang login ``` -* Set the HuggingFace token using the `defang secrets` command +* Set the HuggingFace token using the `defang config` command ```bash -defang secrets set --name HF_TOKEN +defang config set --name HF_TOKEN ``` -[Defang Secrets](../concepts/secrets.md) stores your sensitive information such as API keys, passwords, and other credentials for you. +[Configuration](../concepts/configuration.md) stores your sensitive information such as API keys, passwords, and other credentials for you. ## Step 3: Explore the Compose File -The `compose.yml` file is where you define your services and their configurations. +The `compose.yml` file is where you define your services and their configurations. ### The Mistral Service @@ -85,15 +85,10 @@ services: interval: 5m timeout: 30s retries: 10 - // highlight-start - secrets: - - HF_TOKEN + // highlight-start + environment: + - HF_TOKEN // highlight-end -secrets: - // highlight-start - HF_TOKEN: - external: true - // highlight-end ``` **Let's break it down.** @@ -117,23 +112,18 @@ deploy: - capabilities: ["gpu"] ``` -The Mistral model will be downloaded from HuggingFace. We need to have a HuggingFace Token to enable the installation, so we specify that we need to get the `HF_TOKEN` from our secrets. +The Mistral model will be downloaded from HuggingFace. We need to have a HuggingFace Token to enable the installation, so we specify that we need to get the `HF_TOKEN` configuration value from Defang. + +Specifying the `HF_TOKEN` in the `environment` section of the service in the `compose.yml` file tells Defang to fetch the value from the encrypted configuration store. ```yaml -secrets: +environment: - HF_TOKEN ``` -The `HF_TOKEN` is set as an external secret, which means that the secret is not stored in the compose file but is fetched from the Defang encrypted secrets store. - -```yaml -secrets: - HF_TOKEN: - external: true -``` ### The UI Service -In this sample we also provide a simple UI to interact with the endpoint created by vLLM. The UI service is a Next.js application that runs on port 3000. +In this sample we also provide a simple UI to interact with the endpoint created by vLLM. The UI service is a Next.js application that runs on port 3000. :::tip Networking You can see here how Defang's [networking](../concepts//networking.mdx) works. The `mistral` service is available at `http://mistral:8000`, exactly as it would be in a local `docker-compose` environment. diff --git a/scripts/prep-samples.js b/scripts/prep-samples.js index 5a89b3aac..147b89bbd 100644 --- a/scripts/prep-samples.js +++ b/scripts/prep-samples.js @@ -32,11 +32,13 @@ directories.forEach((category) => { }); }); -const stringified = JSON.stringify(jsonArray); +const stringified = JSON.stringify(jsonArray, null, 2); + +fs.writeFileSync(path.join(__dirname, '..', 'samples.json'), stringified); // we're going to open up the ../docs/samples.md file and replce [] with the stringified JSON const samplesMd = path.join(__dirname, '..', 'docs', 'samples.md'); let samplesMdContents = fs.readFileSync(samplesMd, 'utf8'); -samplesMdContents = samplesMdContents.replace('[]', stringified); +samplesMdContents += ``; fs.writeFileSync(samplesMd, samplesMdContents);