diff --git a/docs/docs/guides/services.md b/docs/docs/guides/services.md new file mode 100644 index 000000000..de72ab433 --- /dev/null +++ b/docs/docs/guides/services.md @@ -0,0 +1,83 @@ +# Services + +A service is an internet-accessible application accessed through a gateway. +These applications can be hosted on both cloud and local backends (behind NAT). + +## Gateways + +To deploy a service, you require a gateway: a small CPU instance with a public IP address and potentially a domain. +Currently, dstack supports AWS, Azure, and GCP backends for creating gateways. +Run the following command to create your first gateway: + +```shell +$ dstack gateway create --project azure + +Creating gateway, it may take some time... + NAME ADDRESS + dstack-gateway-fast-walrus 23.100.30.60 +``` + +You can then use this gateway for services within any project configured on the same hub, +irrespective of the backend type. +Run `dstack gateway list` to view gateways within the project and `dstack gateway delete` to free up resources. + +## Configuring Services + +Construct a service configuration by specifying the gateway hostname and application port in a YAML format: + +```yaml +type: service +gateway: 23.100.30.60 +port: 8000 +commands: + - python -m http.server 8000 +``` + +To avoid directly inputting the hostname in the configuration file, you can use secret interpolation. +Begin by creating a secret in the project where your service will run: + +```shell +$ dstack secrets add GATEWAY_IP 23.100.30.60 --project local +``` + +Then replace the gateway hostname with a variable: + +```yaml +type: service +gateway: ${{ secrets.GATEWAY_IP }} +port: 8000 +commands: + - python -m http.server 8000 +``` + +The default service port is 80, but you can configure a mapping, such as to port 5001: + +```yaml +type: service +gateway: ${{ secrets.GATEWAY_IP }} +port: "5001:8000" +commands: + - python -m http.server 8000 +``` + +## Running Services + +Execute the following command to run a service: + +```shell +$ dstack run . -f service.dstack.yml --project local +``` + +The service will be accessible at `http://23.100.30.60:5001`. + +## Running Multiple Services via the Same Gateway + +Running multiple services with the same gateway hostname and external port simultaneously is not possible. +However, you can reuse the same gateway hostname and external port if a service is no longer active. +If you wish to run multiple services, consider these two solutions: + +- Utilize different ports +- Use distinct domains + +Create a DNS record for your domain, pointing it to the gateway's IP address. +Following this, you can use the domain name as an alias for the gateway hostname. diff --git a/docs/docs/reference/configurations/dev-environment.md b/docs/docs/reference/configurations/dev-environment.md new file mode 100644 index 000000000..0d65f4661 --- /dev/null +++ b/docs/docs/reference/configurations/dev-environment.md @@ -0,0 +1,77 @@ +# type: dev-environment (.dstack.yml) + +Dev environment allows connecting to a remote virtual machine from VS Code Desktop. + +!!! info "Filename" + The configuration file must be named with the suffix `.dstack.yml`. For example, + you can name the configuration file `.dstack.yml` or `dev.dstack.yml`. You can define + these configurations anywhere within your project. + + Each folder may have one default configuration file named `.dstack.yml`. + +## Runtime properties: + +- `ide` - The IDE to run. Currently only `vscode` is supported. +- `env` - (Optional) The list or mapping of environment variables. Interpolation `PATH=/bin/my:$PATH` is supported. +- `ports` - (Optional) The list of exposed ports or mappings. Allowed formats: + - `8000` - expose port `8000` and run the ssh tunnel locally on the same port. + - `3333:8000` - expose port `8000` and run ssh tunnel locally on port `3333`. + - `*:8000` - expose port `8000` and run the ssh tunnel locally on the first available port starting from `8000`. +- `python` - (Optional) The preinstalled major python version (from `3.7` to `3.11`). Mutually exclusive with `image` property. +- `setup` - (Optional) The list of commands to be executed once for the environment. +- `init` - (Optional) The list of commands to be executed every restart of the environment. + +## Custom docker image properties: + +- `image` - (Optional) The docker image. Mutually exclusive with `python` property. +- `entrypoint` - (Optional) The docker entry point to interpret the commands (typically `/bin/bash -i -c`). +- `registry_auth` - (Optional) The credentials to pull the image from the private registry. + - `username` - The username. Supports secrets interpolation `${{ secrets.GHCR_USER }}`. + - `password` - The password or token. Supports secrets interpolation `${{ secrets.GHCR_TOKEN }}`. +- `home_dir` - (Optional) The home directory inside the container. Defaults to `/root`. + +!!! info "Requirements" + Custom docker image must have `sshd`. + You could pre-install `openssh-server` in your `Dockerfile` or use the `build` property. + +## Optimization properties: + +- `cache` - (Optional) The list of directories to cache between the environment restarts. Both absolute and relative paths are supported. +- `build` - (Optional) The list of commands to run during the build stage. You must call `dstack build` first or use the flag `dstack run --build`. + +!!! info "Note" + The build is an experimental feature. Performance gain depends on the type of workload and specific cloud provider. + +## Examples: + +### Minimal configuration + +```yaml +type: dev-environment +ide: vscode +``` + +### Custom docker image + +```yaml +type: dev-environment +ide: vscode +image: ghcr.io/huggingface/text-generation-inference:0.9.3 +build: + - apt-get update + - DEBIAN_FRONTEND=noninteractive apt-get install -y openssh-server + - rm -rf /var/lib/apt/lists/* +env: + - MODEL_ID=meta-llama/Llama-2-7b-chat-hf + - HOSTNAME=0.0.0.0 + - PORT=8080 + # HUGGING_FACE_HUB_TOKEN is stored in secrets +ports: + - 8080 +``` + +[//]: # (TODO: describe profile policies defaults) + +[//]: # (TODO: Add examples) + +[//]: # (TODO: Mention here or somewhere else of how it works. What base image is used, how ports are forwarded, etc.) diff --git a/docs/docs/reference/configurations/service.md b/docs/docs/reference/configurations/service.md new file mode 100644 index 000000000..b32cca4fc --- /dev/null +++ b/docs/docs/reference/configurations/service.md @@ -0,0 +1,56 @@ +# type: service (.dstack.yml) + +Service allows running an application and exposing it over HTTP to the global internet through the gateway. + +!!! info "Filename" + The configuration file must be named with the suffix `.dstack.yml`. For example, + you can name the configuration file `.dstack.yml` or `service.dstack.yml`. You can define + these configurations anywhere within your project. + + Each folder may have one default configuration file named `.dstack.yml`. + +## Runtime properties: + +- `commands` - The list of commands to be executed every restart of the environment. +- `gateway` - The gateway IP address or domain. Supports secrets interpolation `${{ secrets.GATEWAY_IP }}`. +- `port` - The exposed port or mapping. Allowed formats: + - `8000` - expose port `8000` at gateway's port `80`. + - `3333:8000` - expose port `8000` at gateway's port `3333`. +- `env` - (Optional) The list or mapping of environment variables. Interpolation `PATH=/bin/my:$PATH` is supported. +- `python` - (Optional) The preinstalled major python version (from `3.7` to `3.11`). Mutually exclusive with `image` property. +- `setup` - (Optional) The list of commands to be executed once for the environment. + +## Custom docker image properties: + +- `image` - (Optional) The docker image. Mutually exclusive with `python` property. +- `entrypoint` - (Optional) The docker entry point to interpret the commands (typically `/bin/bash -i -c`). +- `registry_auth` - (Optional) The credentials to pull the image from the private registry. + - `username` - The username. Supports secrets interpolation `${{ secrets.GHCR_USER }}`. + - `password` - The password or token. Supports secrets interpolation `${{ secrets.GHCR_TOKEN }}`. +- `home_dir` - (Optional) The home directory inside the container. Defaults to `/root`. + +## Optimization properties: + +- `cache` - (Optional) The list of directories to cache between the environment restarts. Both absolute and relative paths are supported. +- `build` - (Optional) The list of commands to run during the build stage. You must call `dstack build` first or use the flag `dstack run --build`. + +!!! info "Note" + The build is an experimental feature. Performance gain depends on the type of workload and specific cloud provider. + +## Examples: + +### Simple web server + +```yaml +type: service +gateway: ${{ secrets.GATEWAY_IP }} +port: 8000 +commands: + - python -m http.server 8000 +``` + +[//]: # (TODO: describe profile policies defaults) + +[//]: # (TODO: Add examples) + +[//]: # (TODO: Mention here or somewhere else of how it works. What base image is used, how ports are forwarded, etc.) diff --git a/docs/docs/reference/configurations/task.md b/docs/docs/reference/configurations/task.md new file mode 100644 index 000000000..c0a70a3fd --- /dev/null +++ b/docs/docs/reference/configurations/task.md @@ -0,0 +1,64 @@ +# type: task (.dstack.yml) + +The task allows to run commands non-interactively: from fine-tuning to serving. + +!!! info "Filename" + The configuration file must be named with the suffix `.dstack.yml`. For example, + you can name the configuration file `.dstack.yml` or `task.dstack.yml`. You can define + these configurations anywhere within your project. + + Each folder may have one default configuration file named `.dstack.yml`. + +## Runtime properties: + +- `commands` - The list of commands to be executed every restart of the environment. +- `env` - (Optional) The list or mapping of environment variables. Interpolation `PATH=/bin/my:$PATH` is supported. +- `ports` - (Optional) The list of exposed ports or mappings. Allowed formats: + - `8000` - expose port `8000` and run the ssh tunnel locally on the same port. + - `3333:8000` - expose port `8000` and run ssh tunnel locally on port `3333`. + - `*:8000` - expose port `8000` and run the ssh tunnel locally on the first available port starting from `8000`. +- `python` - (Optional) The preinstalled major python version (from `3.7` to `3.11`). Mutually exclusive with `image` property. +- `setup` - (Optional) The list of commands to be executed once for the environment. + +## Custom docker image properties: + +- `image` - (Optional) The docker image. Mutually exclusive with `python` property. +- `entrypoint` - (Optional) The docker entry point to interpret the commands (typically `/bin/bash -i -c`). +- `registry_auth` - (Optional) The credentials to pull the image from the private registry. + - `username` - The username. Supports secrets interpolation `${{ secrets.GHCR_USER }}`. + - `password` - The password or token. Supports secrets interpolation `${{ secrets.GHCR_TOKEN }}`. +- `home_dir` - (Optional) The home directory inside the container. Defaults to `/root`. + +## Optimization properties: + +- `cache` - (Optional) The list of directories to cache between the environment restarts. Both absolute and relative paths are supported. +- `build` - (Optional) The list of commands to run during the build stage. You must call `dstack build` first or use the flag `dstack run --build`. + +!!! info "Note" + The build is an experimental feature. Performance gain depends on the type of workload and specific cloud provider. + +## Examples: + +### Hello world + +```yaml +type: task +commands: + - echo Hello world! +``` + +### Run custom python script + +```yaml +type: task +build: + - pip install --no-cache-dir -r requirements.txt +commands: + - python train.py ${{ run.args }} +``` + +[//]: # (TODO: describe profile policies defaults) + +[//]: # (TODO: Add examples) + +[//]: # (TODO: Mention here or somewhere else of how it works. What base image is used, how ports are forwarded, etc.) diff --git a/docs/docs/reference/dstack.yml.md b/docs/docs/reference/dstack.yml.md index 41d903191..6258961c7 100644 --- a/docs/docs/reference/dstack.yml.md +++ b/docs/docs/reference/dstack.yml.md @@ -10,29 +10,8 @@ types: `dev-environment`, `task`, and `service`. Each folder may have one default configuration file named `.dstack.yml`. -Below is a full reference of all available properties. +Read more about configuration types: -- `build` - (Optional) The list of bash commands to build the environment. -- `cache` - (Optional) The directories to be cached between runs. -- `commands` - (Required if `type` is `task`). The list of bash commands to run as a task. -- `entrypoint` - (Optional) The Docker entrypoint. -- `env` - (Optional) The mapping or the list of environment variables (e.g. `PYTHONPATH: src` or `PYTHONPATH=src`). -- `gateway` - (Required if `type` is `service`) Gateway IP address or domain name. -- `ide` - (Required if `type` is `dev-environment`). Can be `vscode`. -- `image` - (Optional) The name of the Docker image. -- `init` - (Optional, only for `dev-environment` type) The list of bash commands to execute on each run. -- `port` - (Required) The service port to expose (only for `service`) -- `ports` - (Optional) The list of port numbers to expose (only for `dev-environment` and `task`). -- `python` - (Optional) The major version of Python to pre-install (e.g., `"3.11"`). Defaults to the current version installed locally. Mutually exclusive with `image`. -- `registry_auth` - (Optional) Credentials to pull the private Docker image. - - `password` - (Required) Password or access token. - - `username` - (Required) Username. -- `type` - (Required) The type of the configurations. Can be `dev-environment`, `task`, or `service`. - -[//]: # (- `home_dir` - (Optional) The absolute path to the home directory inside the container) - -[//]: # (TODO: `artifacts` aren't documented) - -[//]: # (TODO: Add examples) - -[//]: # (TODO: Mention here or somewhere else of how it works. What base image is used, how ports are forwarded, etc.) +* type: [dev-environment](configurations/dev-environment.md) +* type: [task](configurations/task.md) +* type: [service](configurations/service.md) diff --git a/mkdocs.yml b/mkdocs.yml index c808e91ba..8f496a492 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -146,10 +146,14 @@ nav: - Guides: - Dev environments: docs/guides/dev-environments.md - Tasks: docs/guides/tasks.md + - Services: docs/guides/services.md # - Artifacts: docs/guides/artifacts.md - Projects: docs/guides/projects.md - Reference: - - .dstack.yml: docs/reference/dstack.yml.md + - .dstack.yml: + - "type: dev-environment": docs/reference/configurations/dev-environment.md + - "type: task": docs/reference/configurations/task.md + - "type: service": docs/reference/configurations/service.md - profiles.yml: docs/reference/profiles.yml.md - CLI: - dstack run: docs/reference/cli/run.md