Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions docs/cli/Guides/deploy-app/deploy-app-example.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ RUN chmod +x /usr/local/bin/usd_to_crypto.py
COPY entrypoint.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh

# Set /sp as workdir (doesn't matter in this case; entrypoint.sh uses /sp/output as workdir)
# Set /sp as workdir (doesn't matter in this case -- entrypoint.sh uses /sp/output as workdir)
WORKDIR /sp

# Set entrypoint
Expand Down Expand Up @@ -137,8 +137,8 @@ tar -czvf input.tar.gz ./input.txt

```shell
./spctl files upload ./input.tar.gz \
--filename input.tar.gz \
--output input.resource.json
--filename input.tar.gz \
--output input.resource.json
```

### 3. Deploy
Expand All @@ -147,9 +147,9 @@ Place an order:

```shell
./spctl workflows create \
--tee 7 \
--solution ./usd_to_crypto.resource.json \
--data ./input.resource.json
--tee 7 \
--solution ./usd_to_crypto.resource.json \
--data ./input.resource.json
```

Find the order ID in the output, for example:
Expand Down
183 changes: 183 additions & 0 deletions docs/cli/Guides/swarm-vllm.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
---
id: "swarm-vllm"
title: "vLLM on Super Swarm"
slug: "/guides/swarm-vllm"
sidebar_position: 20
---

This guide provides step-by-step instructions for deploying MedGemma and Apertus on Super Swarm using vLLM.

## Prerequisites

- [kubectl](https://kubernetes.io/docs/tasks/tools/)
- [helm](https://helm.sh/docs/intro/install/)
- A domain
- For [MedGemma](https://huggingface.co/google/medgemma-1.5-4b-it): a Hugging Face token from an account that has already accepted the model's terms

Also, download and rename deployment scripts:

- [`deploy_medgemma_official.sh`](/files/deploy_medgemma_official.sh)
- [`deploy_apertus_official.sh`](/files/deploy_apertus_official.sh)

## 1. Sign in to Super Swarm

In the Super Swarm dashboard, sign in using MetaMask:

<img src={require('../images/swarm-log-in.png').default} width="auto" height="auto" border="1"/>
<br/>

## 2. Create a Kubernetes cluster

2.1. Go to **Kubernetes** and press **Create Cluster**:

<img src={require('../images/kubernetes-create-cluster.png').default} width="auto" height="auto" border="1"/>
<br/>
<br/>

2.2. Add a GPU to the cluster, allocate resources, and press **Create Cluster**:

<img src={require('../images/create-kubernetes-space.png').default} width="auto" height="auto" border="1"/>
<br/>

## 3. Download the cluster configuration file

<img src={require('../images/kubernetes-download-kubeconfig.png').default} width="auto" height="auto" border="1"/>
<br/>

## 4. Point `kubectl` to the configuration file

Execute the following command:

```shell
export KUBECONFIG=<xxxx-xxx-xxx-xxxx>-kubeconfig.yaml
```

Replace `<xxxx-xxx-xxx-xxxx>-kubeconfig.yaml` with the name of the downloaded configuration file.

## 5. Update the scripts

In both scripts (`deploy_medgemma_official.sh` and `deploy_apertus_official.sh`), find `BASE_DOMAIN="${BASE_DOMAIN:-monai-swarm.win}"` and replace `monai-swarm.win` with your domain.

## 6. Set the API key

Choose any password that will protect your API endpoints. Execute the following command and type your chosen secret (characters won't be displayed):

```shell
read -rs API_KEY && export API_KEY
```

## 7. Deploy the model

### Apertus

```shell
bash deploy_apertus_official.sh
```

The deployment usually takes 5-7 minutes.

A working Apertus config is already set in the script:

```
dtype=bfloat16
max-model-len=32768
gpu-memory-utilization=0.55
max-num-seqs=8
max-num-batched-tokens=4096
```

### MedGemma

```shell
export HF_TOKEN=hf_xxx
bash deploy_medgemma_official.sh
```

Replace `hf_xxx` with an HF_TOKEN.

Alternatively, create a `.hf_token` file with the token next to `deploy_medgemma_official.sh`; the script will read it automatically.

A working MedGemma config is already set in the script:

```
dtype=bfloat16
max-model-len=8192
gpu-memory-utilization=0.40
--mm-processor-cache-gb 1
max-num-seqs=4
max-num-batched-tokens=2048
```

## 8. Check Kubernetes

```shell
kubectl get pods -o wide
kubectl get svc
kubectl get ingress
```

Expected output:

- Two pods in `1/1 Running`
- Two services
- Two ingresses

## 9. Confirm DNS records

Back in the Super Swarm dashboard, go to **Ingresses** and note the two hostnames listed there.

<img src={require('../images/ingresses.png').default} width="auto" height="auto" border="1"/>
<br/>
<br/>

For each hostname, add a CNAME record pointing to it and a TXT record for domain verification at your DNS provider.

## 10. Publish the cluster

In the Super Swarm dashboard, go to **Kubernetes** and publish the cluster.

<img src={require('../images/kubernetes-publish-cluster.png').default} width="auto" height="auto" border="1"/>
<br/>

## 11. Send test requests

In the test requests below, replace:

- `<DOMAIN>` with your domain.
- `<API_KEY>` with the key you set in [Step 6](/cli/guides/swarm-vllm#6-set-the-api-key).

### Apertus

```shell
curl https://apertus-vllm.<DOMAIN>/v1/completions \
-H 'Authorization: Bearer <API_KEY>' \
-H 'Content-Type: application/json' \
-d '{
"model": "swiss-ai/Apertus-8B-2509",
"prompt": "Write a concise technical summary of Kubernetes GPU scheduling.",
"temperature": 0,
"max_tokens": 200
}'
```

### MedGemma

```shell
curl https://medgemma-vllm.<DOMAIN>/v1/chat/completions \
-H 'Authorization: Bearer <API_KEY>' \
-H 'Content-Type: application/json' \
-d '{
"model": "google/medgemma-1.5-4b-it",
"messages": [
{
"role": "user",
"content": [
{"type": "text", "text": "Describe this image briefly."},
{"type": "image_url", "image_url": {"url": "data:image/png;base64,PASTE_BASE64_HERE"}}
]
}
],
"temperature": 0,
"max_tokens": 120
}'
```
Binary file added docs/cli/images/create-kubernetes-space.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/cli/images/ingresses.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/cli/images/kubernetes-create-cluster.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/cli/images/kubernetes-publish-cluster.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/cli/images/swarm-log-in.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
8 changes: 3 additions & 5 deletions docs/marketplace/Guides/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,12 @@ sidebar_position: 6

This guide provides step-by-step instructions on how to set up your personal Storj account.

The guide is intended for advanced Web3 users; feel free to skip it and continue using the default recommended option—**Super Protocol cloud**. Read about [types of storage](/marketplace/account/web3#storage).
The guide is intended for advanced users; feel free to skip it and continue using the default recommended option—**Super Protocol cloud**. Read about [types of storage](/marketplace/account#storage).

<img src={require('../images/web3-account.png').default} width="auto" height="auto" border="1"/>
<img src={require('../images/web2-account.png').default} width="auto" height="auto" border="1"/>
<br/>
<br/>

Web2 users must first [log in as a Web3 user](/marketplace/guides/log-in) to be able to upload to a personal Storj account instead of the Super Protocol cloud.

## Step 1. Register a Storj account

If you don't already have a [Storj](https://www.storj.io/) account, register one. Both free Trial and Pro accounts are suitable. Note that with a Trial account, your files will become inaccessible once the trial period ends.
Expand All @@ -36,7 +34,7 @@ As a result, you should have two pairs Access Key + Secret Key.

## Step 4. Set up your Super Protocol Web3 account

Open the [Marketplace web app](https://marketplace.superprotocol.com/). Log in as a Web3 user and open the **Account** window.
Open the [Marketplace web app](https://marketplace.superprotocol.com/), sign in and open the **Account** window.

<img src={require('../images/marketplace-mm-account-menu.png').default} width="auto" height="auto" border="1"/>
<br/>
Expand Down
52 changes: 52 additions & 0 deletions docs/marketplace/account.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
id: "account"
title: "Enter Marketplace"
slug: "/account"
sidebar_position: 3
---

Super Protocol supports two login methods:

- Web2 requires an account on one of the supported platforms:
- Google
- Hugging Face
- GitHub
- Microsoft
- Web3 requires a software wallet installed as a browser extension:
- MetaMask
- Trust Wallet

For instructions on how to set up software wallets and connect them to the Marketplace, read [How to Log In as a Web3 User](/marketplace/guides/log-in).

<img src={require('./images/enter-marketplace.png').default} width="auto" height="auto" border="1"/>
<br/>
<br/>

## Account window

This window shows your user account settings.

<img src={require('./images/web2-account.png').default} width="auto" height="auto" border="1"/>
<br/>
<br/>

**User ID**: your unique user ID.

**Login**: the OAuth2 provider and your login email address.

The **Get SPPI** button allows you to get tokens necessary to place <a id="order"><span className="dashed-underline">orders</span></a>.

### Storage

You have two options of decentralized storage to upload files:

- **Super Protocol cloud**:
- Recommended for most users.
- Does not require additional setup.
- Uses Super Protocol's Storj account and thus relies on Super Protocol as the storage provider.
- **Your Storj account**:
- Intended for advanced users.
- Requires creating and [setting up a Storj account](/marketplace/guides/storage).
- Gives sole control over the uploaded content and storage account.

Read [How to Set Up Storage](/marketplace/guides/storage) for step-by-step instructions.
39 changes: 0 additions & 39 deletions docs/marketplace/account/index.md

This file was deleted.

31 changes: 0 additions & 31 deletions docs/marketplace/account/web2.md

This file was deleted.

Loading
Loading