Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ Using a Private Network for communications between your Instances hosting your a
import requests

PAYLOAD = {
"model": "<MODEL_DEPLOYED>", # EXAMPLE= meta/llama-3-8b-instruct:bf16
"model": "<MODEL_DEPLOYED>", # EXAMPLE= meta/llama-3.1-8b-instruct:fp8
"messages": [
{"role": "system",
"content": "You are a helpful, respectful and honest assistant."},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ categories:
| Attribute | Details |
|-----------------|------------------------------------|
| Provider | [Meta](https://llama.meta.com/llama3/) |
| Model Name | `llama-3-70b-instruct` |
| Compatible Instances | H100 (FP8) |
| Context size | 8192 tokens |

Expand Down Expand Up @@ -62,7 +61,7 @@ curl -s \
-H "Content-Type: application/json" \
--request POST \
--url "https://<Deployment UUID>.ifr.fr-par.scaleway.com/v1/chat/completions" \
--data '{"model":"llama-3-70b-instruct", "messages":[{"role": "user","content": "Sing me a song about Xavier Niel"}], "max_tokens": 500, "top_p": 1, "temperature": 0.7, "stream": false}'
--data '{"model":"meta/llama-3-70b-instruct:fp8", "messages":[{"role": "user","content": "Sing me a song about Xavier Niel"}], "max_tokens": 500, "top_p": 1, "temperature": 0.7, "stream": false}'
```

Make sure to replace `<IAM API key>` and `<Deployment UUID>` with your actual [IAM API key](/identity-and-access-management/iam/how-to/create-api-keys/) and the Deployment UUID you are targeting.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ categories:
| Attribute | Details |
|-----------------|------------------------------------|
| Provider | [Meta](https://llama.meta.com/llama3/) |
| Model Name | `llama-3-8b-instruct` |
| Compatible Instances | L4, H100 (FP8, BF16) |
| Context size | 8192 tokens |

Expand Down Expand Up @@ -66,7 +65,7 @@ curl -s \
-H "Content-Type: application/json" \
--request POST \
--url "https://<Deployment UUID>.ifr.fr-par.scaleway.com/v1/chat/completions" \
--data '{"model":"llama-3-8b-instruct", "messages":[{"role": "user","content": "There is a llama in my garden, what should I do?"}], "max_tokens": 500, "top_p": 1, "temperature": 0.7, "stream": false}'
--data '{"model":"meta/llama-3-8b-instruct:fp8", "messages":[{"role": "user","content": "There is a llama in my garden, what should I do?"}], "max_tokens": 500, "top_p": 1, "temperature": 0.7, "stream": false}'
```

Make sure to replace `<IAM API key>` and `<Deployment UUID>` with your actual [IAM API key](/identity-and-access-management/iam/how-to/create-api-keys/) and the Deployment UUID you are targeting.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,7 @@ categories:
| Attribute | Details |
|-----------------|------------------------------------|
| Provider | [Meta](https://llama.meta.com/llama3/) |
| License | [Llama 3.1 community](https://llama.meta.com/llama3_1/license/) |
| Model Name | `llama-3.1-70b-instruct` |
| License | [Llama 3.1 community](https://llama.meta.com/llama3_1/license/) | |
| Compatible Instances | H100 (FP8), H100-2 (FP8, BF16) |
| Context Length | up to 128k tokens |

Expand Down Expand Up @@ -61,7 +60,7 @@ curl -s \
-H "Content-Type: application/json" \
--request POST \
--url "https://<Deployment UUID>.ifr.fr-par.scaleway.com/v1/chat/completions" \
--data '{"model":"llama-3.1-70b-instruct", "messages":[{"role": "user","content": "There is a llama in my garden, what should I do?"}], "max_tokens": 500, "temperature": 0.7, "stream": false}'
--data '{"model":"meta/llama-3.1-70b-instruct:fp8", "messages":[{"role": "user","content": "There is a llama in my garden, what should I do?"}], "max_tokens": 500, "temperature": 0.7, "stream": false}'
```

Make sure to replace `<IAM API key>` and `<Deployment UUID>` with your actual [IAM API key](/identity-and-access-management/iam/how-to/create-api-keys/) and the Deployment UUID you are targeting.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ categories:
|-----------------|------------------------------------|
| Provider | [Meta](https://llama.meta.com/llama3/) |
| License | [Llama 3.1 community](https://llama.meta.com/llama3_1/license/) |
| Model Name | `llama-3.1-8b-instruct` |
| Compatible Instances | L4, H100, H100-2 (FP8, BF16) |
| Context Length | up to 128k tokens |

Expand Down Expand Up @@ -62,7 +61,7 @@ curl -s \
-H "Content-Type: application/json" \
--request POST \
--url "https://<Deployment UUID>.ifr.fr-par.scaleway.com/v1/chat/completions" \
--data '{"model":"llama-3.1-8b-instruct", "messages":[{"role": "user","content": "There is a llama in my garden, what should I do?"}], "max_tokens": 500, "temperature": 0.7, "stream": false}'
--data '{"model":"meta/llama-3.1-8b-instruct:fp8", "messages":[{"role": "user","content": "There is a llama in my garden, what should I do?"}], "max_tokens": 500, "temperature": 0.7, "stream": false}'
```

Make sure to replace `<IAM API key>` and `<Deployment UUID>` with your actual [IAM API key](/identity-and-access-management/iam/how-to/create-api-keys/) and the Deployment UUID you are targeting.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,13 @@ categories:
| Attribute | Details |
|-----------------|------------------------------------|
| Provider | [Mistral](https://mistral.ai/technology/#models) |
| Model Name | `mistral-7b-instruct-v0.3` |
| Compatible Instances | L4 (BF16) |
| Context size | 32K tokens |

## Model name

```bash
mistral-7b-instruct-v0.3:bf16
mistral/mistral-7b-instruct-v0.3:bf16
```

## Compatible Instances
Expand Down Expand Up @@ -55,7 +54,7 @@ curl -s \
-H "Content-Type: application/json" \
--request POST \
--url "https://<Deployment UUID>.ifr.fr-par.scaleway.com/v1/chat/completions" \
--data '{"model":"mistral-7b-instruct-v0.3", "messages":[{"role": "user","content": "Explain Public Cloud in a nutshell."}], "top_p": 1, "temperature": 0.7, "stream": false}'
--data '{"model":"mistral/mistral-7b-instruct-v0.3:bf16", "messages":[{"role": "user","content": "Explain Public Cloud in a nutshell."}], "top_p": 1, "temperature": 0.7, "stream": false}'
```

Make sure to replace `<IAM API key>` and `<Deployment UUID>` with your actual [IAM API key](/identity-and-access-management/iam/how-to/create-api-keys/) and the Deployment UUID you are targeting.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,13 @@ categories:
| Attribute | Details |
|-----------------|------------------------------------|
| Provider | [Mistral](https://mistral.ai/technology/#models) |
| Model Name | `mistral-nemo-instruct-2407` |
| Compatible Instances | H100 (FP8) |
| Context size | 128K tokens |

## Model name

```bash
mistral-nemo-instruct-2407:fp8
mistral/mistral-nemo-instruct-2407:fp8
```

## Compatible Instances
Expand Down Expand Up @@ -61,7 +60,7 @@ curl -s \
-H "Content-Type: application/json" \
--request POST \
--url "https://<Deployment UUID>.ifr.fr-par.scaleway.com/v1/chat/completions" \
--data '{"model":"mistral-nemo-instruct-2407", "messages":[{"role": "user","content": "Sing me a song about Xavier Niel"}], "top_p": 1, "temperature": 0.35, "stream": false}'
--data '{"model":"mistral/mistral-nemo-instruct-2407:fp8", "messages":[{"role": "user","content": "Sing me a song about Xavier Niel"}], "top_p": 1, "temperature": 0.35, "stream": false}'
```

Make sure to replace `<IAM API key>` and `<Deployment UUID>` with your actual [IAM API key](/identity-and-access-management/iam/how-to/create-api-keys/) and the Deployment UUID you are targeting.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ categories:
| Attribute | Details |
|-----------------|------------------------------------|
| Provider | [Mistral](https://mistral.ai/technology/#models) |
| Model Name | `mixtral-8x7b-instruct-v0.1` |
| Compatible Instances | H100 (FP8) - H100-2 (FP16) |
| Context size | 32k tokens |

Expand Down Expand Up @@ -57,7 +56,7 @@ curl -s \
-H "Content-Type: application/json" \
--request POST \
--url "https://<Deployment UUID>.ifr.fr-par.scaleway.com/v1/chat/completions" \
--data '{"model":"mixtral-8x7b-instruct-v0.1", "messages":[{"role": "user","content": "Sing me a song about Scaleway"}], "max_tokens": 200, "top_p": 1, "temperature": 1, "stream": false}'
--data '{"model":"mistral/mixtral-8x7b-instruct-v0.1:fp8", "messages":[{"role": "user","content": "Sing me a song about Scaleway"}], "max_tokens": 200, "top_p": 1, "temperature": 1, "stream": false}'
```

Make sure to replace `<IAM API key>` and `<Deployment UUID>` with your actual [IAM API key](/identity-and-access-management/iam/how-to/create-api-keys/) and the Deployment UUID you are targeting.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ categories:
| Attribute | Details |
|-----------------|------------------------------------|
| Provider | [Mistral](https://mistral.ai/technology/#models) |
| Model Name | `pixtral-12b-2409` |
| Compatible Instances | H100, H100-2 (bf16) |
| Context size | 128k tokens |

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,10 @@ categories:
| Attribute | Details |
|-----------------|------------------------------------|
| Provider | [sentence-transformers](https://www.sbert.net/) |
| Model Name | `sentence-t5-xxl` |
| Compatible Instances | L4 (FP32) |
| Context size | 512 tokens |

## Model names
## Model name

```bash
sentence-transformers/sentence-t5-xxl:fp32
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ categories:
| Attribute | Details |
|-----------------|------------------------------------|
| Provider | [WizardLM](https://wizardlm.github.io/) |
| Model Name | `wizardlm-70B-V1.0` |
| Compatible Instances | H100 (FP8) - H100-2 (FP16) |
| Context size | 4,096 tokens |

Expand Down Expand Up @@ -55,7 +54,7 @@ curl -s \
-H "Content-Type: application/json" \
--request POST \
--url "https://<Deployment UUID>.ifr.fr-par.scaleway.com/v1/chat/completions" \
--data '{"model":"wizardlm-70B-V1.0", "messages":[{"role": "user","content": "Say hello to Scaleway's Inference"}], "max_tokens": 200, "top_p": 1, "temperature": 1, "stream": false}'
--data '{"model":"wizardlm/wizardlm-70b-v1.0:fp8", "messages":[{"role": "user","content": "Say hello to Scaleway's Inference"}], "max_tokens": 200, "top_p": 1, "temperature": 1, "stream": false}'
```

Make sure to replace `<IAM API key>` and `<Deployment UUID>` with your actual [IAM API key](/identity-and-access-management/iam/how-to/create-api-keys/) and the Deployment UUID you are targeting.
Expand Down