Skip to content

Commit

Permalink
Add OCI docs (#1312)
Browse files Browse the repository at this point in the history
  • Loading branch information
jvstme committed Jun 5, 2024
1 parent 9d88f46 commit 4492317
Show file tree
Hide file tree
Showing 6 changed files with 120 additions and 9 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Orchestrate AI workloads in any cloud

`dstack` is an open-source container orchestration engine designed for AI workloads across any cloud or data center.

The supported cloud providers include AWS, GCP, Azure, Lambda, TensorDock, Vast.ai, CUDO, and RunPod.
The supported cloud providers include AWS, GCP, Azure, Lambda, OCI, TensorDock, Vast.ai, CUDO, and RunPod.
You can also use `dstack` ro run workloads on on-prem servers.

## Latest news ✨
Expand All @@ -49,7 +49,7 @@ pip install "dstack[all]" -U

### Configure backends

If you have default AWS, GCP, or Azure credentials on your machine, the `dstack` server will pick them up automatically.
If you have default AWS, GCP, Azure, or OCI credentials on your machine, the `dstack` server will pick them up automatically.

Otherwise, you need to manually specify the cloud credentials in `~/.dstack/server/config.yml`.

Expand Down
5 changes: 5 additions & 0 deletions contributing/GPUHUNT.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,11 @@ Some providers offer extreme flexibility in possible configurations, but not all
- Queries all known configurations
- Adds all known regions to all configurations

### OCI

- Parses Oracle's [Cost Estimator](https://www.oracle.com/cloud/costestimator.html) datasets
- Duplicates each offer in all regions, since prices are the same everywhere and availability is mostly the same

### Nebius

- Uses hardcoded CPU and GPU platforms configurations
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

`dstack` is an open-source container orchestration engine designed for running AI workloads across any cloud or data center.

> The supported cloud providers include AWS, GCP, Azure, Lambda, TensorDock, Vast.ai, CUDO, and RunPod.
> The supported cloud providers include AWS, GCP, Azure, Lambda, OCI, TensorDock, Vast.ai, CUDO, and RunPod.
> You can also use `dstack` ro run workloads on on-prem servers.
`dstack` supports dev environements, running tasks on clusters, and deployment with auto-scaling and
Expand Down
12 changes: 10 additions & 2 deletions docs/docs/protips.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ Examples:

## Service quotas

If you're using your own AWS, GCP, or Azure accounts, before you can use GPUs or spot instances, you have to request the
If you're using your own AWS, GCP, Azure, or OCI accounts, before you can use GPUs or spot instances, you have to request the
corresponding service quotas for each type of instance in each region.

??? info "AWS"
Expand Down Expand Up @@ -207,7 +207,7 @@ corresponding service quotas for each type of instance in each region.
- `Preemtible H100 GPUs` (spot H100)

??? info "Azure"
Check this [guide :material-arrow-top-right-thin:{ .external }](https://learn.microsoft.com/en-us/azure/quotas/quickstart-increase-quota-portal){:target="_blank"} on Compute Engine service quotas.
Check this [guide :material-arrow-top-right-thin:{ .external }](https://learn.microsoft.com/en-us/azure/quotas/quickstart-increase-quota-portal){:target="_blank"} on Azure service quotas.
The relevant service quotas include:

- `Total Regional Spot vCPUs` (any spot instances)
Expand All @@ -219,6 +219,14 @@ corresponding service quotas for each type of instance in each region.
- `Standard NCadsH100v5 Family vCPUs` (on-demand H100)
- `Standard NDSH100v5 Family vCPUs` (on-demand H100 x8)

??? info "OCI"
Check this [guide :material-arrow-top-right-thin:{ .external }](https://docs.oracle.com/en-us/iaas/Content/General/Concepts/servicelimits.htm#Requesti){:target="_blank"} on requesting OCI service limits increase.
The relevant service category is compute. The relevant resources include:

- `GPUs for GPU.A10 based VM and BM instances` (on-demand A10)
- `GPUs for GPU2 based VM and BM instances` (on-demand P100)
- `GPUs for GPU3 based VM and BM instances` (on-demand V100)

Note, for AWS, GCP, and Azure, service quota values are measured with the number of CPUs rather than GPUs.

## Data and models
Expand Down
87 changes: 87 additions & 0 deletions docs/docs/reference/server/config.yml.md
Original file line number Diff line number Diff line change
Expand Up @@ -371,6 +371,68 @@ projects:

</div>

### OCI

There are two ways to configure OCI: using client credentials or using the default credentials.

=== "Client credentials"

To create client credentials, open the [OCI Console :material-arrow-top-right-thin:{ .external }](https://cloud.oracle.com), click the Profile icon in the top bar, go to My Profile, API Keys. Add a new key by clicking "Add API key".

After you add a key, the Console will show a configuration file preview. Copy the settings from the preview to `dstack` configuration as shown below. Paste the contents of the private key file in the `key_content` property.

<div editor-title="~/.dstack/server/config.yml">

```yaml
projects:
- name: main
backends:
- type: oci
creds:
type: client
user: ocid1.user.oc1..aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
tenancy: ocid1.tenancy.oc1..aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
region: eu-frankfurt-1
fingerprint: 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
key_content: |
-----BEGIN PRIVATE KEY-----
your-oci-api-key
-----END PRIVATE KEY-----
```

</div>

=== "Default credentials"

If you already have a configuration file for the OCI CLI, `dstack` can pick the credentials from it automatically.

??? info "How to create the OCI CLI config file"
Open the [OCI Console :material-arrow-top-right-thin:{ .external }](https://cloud.oracle.com), click the Profile icon in the top bar, go to My Profile, API Keys. Add a new key by clicking "Add API key".

After you add a key, the Console will show a configuration file preview. Create the `~/.oci/config` text file and save the contents of the preview there. Configure the path to your API key as suggested in the preview.

<div editor-title="~/.dstack/server/config.yml">

```yaml
projects:
- name: main
backends:
- type: oci
creds:
type: default
```

</div>

!!! info "NOTE:"
If you have just added your API key in the OCI Console, you may see errors when running `dstack server`, as it takes some time for the key to become fully operational. Try again in a few minutes.

??? info "Required OCI permissions"
`dstack` is guaranteed to work with administrator permissions. An example of a more restrictive policy will be added later.
```
ALLOW GROUP Administrators to manage all-resources IN TENANCY
```

### TensorDock

Log into your [TensorDock :material-arrow-top-right-thin:{ .external }](https://marketplace.tensordock.com/) account, click API in the sidebar, and use the `Create an Authorization`
Expand Down Expand Up @@ -700,6 +762,31 @@ In case of a self-managed cluster, also specify the IP address of any node in th
type:
required: true

## `projects[n].backends[type=oci]` { #oci data-toc-label="backends[type=oci]" }

#SCHEMA# dstack._internal.server.services.config.OCIConfig
overrides:
show_root_heading: false
type:
required: true
item_id_prefix: oci-

## `projects[n].backends[type=oci].creds` { #oci-creds data-toc-label="backends[type=oci].creds" }

=== "Client"
#SCHEMA# dstack._internal.core.models.backends.oci.OCIClientCreds
overrides:
show_root_heading: false
type:
required: true

=== "Default"
#SCHEMA# dstack._internal.core.models.backends.oci.OCIDefaultCreds
overrides:
show_root_heading: false
type:
required: true

## `projects[n].backends[type=tensordock]` { #tensordock data-toc-label="backends[type=tensordock]" }

#SCHEMA# dstack._internal.server.services.config.TensorDockConfig
Expand Down
19 changes: 15 additions & 4 deletions src/dstack/_internal/server/services/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -245,10 +245,21 @@ class NebiusAPIConfig(CoreModel):


class OCIConfig(CoreModel):
type: Literal["oci"] = "oci"
creds: Optional[AnyOCICreds]
regions: Optional[List[str]] = None
compartment_id: Optional[str] = None
type: Annotated[Literal["oci"], Field(description="The type of backend")] = "oci"
creds: Annotated[AnyOCICreds, Field(description="The credentials")]
regions: Annotated[
Optional[List[str]],
Field(description="List of region names for running dstack jobs. Omit to use all regions"),
] = None
compartment_id: Annotated[
Optional[str],
Field(
description=(
"Compartment where dstack will create all resources. "
"Omit to instruct dstack to create a new compartment"
)
),
] = None


class RunpodConfig(CoreModel):
Expand Down

0 comments on commit 4492317

Please sign in to comment.