Skip to content

Commit

Permalink
docs: update jcloud gpu usage (#809)
Browse files Browse the repository at this point in the history
* docs: update jcloud gpu usage

* chore: housekeeping

* docs: commit suggestion

Co-authored-by: felix-wang <35718120+numb3r3@users.noreply.github.com>

* docs: add gpu yaml sample

* docs: use latest gpu tag

Co-authored-by: felix-wang <35718120+numb3r3@users.noreply.github.com>

* docs: use latest gpu tag

Co-authored-by: felix-wang <35718120+numb3r3@users.noreply.github.com>
  • Loading branch information
ZiniuYu and numb3r3 committed Aug 30, 2022
1 parent 59b9f77 commit a5893c7
Showing 1 changed file with 19 additions and 6 deletions.
25 changes: 19 additions & 6 deletions docs/hosting/on-jcloud.md
@@ -1,9 +1,5 @@
# Host on JCloud

```{warning}
JCloud does not support GPU hosting yet. Hence `clip_server` deployed on JCloud will be run on CPU.
```

Essentially `clip_server` is a Jina [Flow](https://docs.jina.ai/fundamentals/flow/). Any Jina Flow can be hosted on [JCloud](https://docs.jina.ai/fundamentals/jcloud/), hence `clip_server` can be hosted on JCloud as well. Learn more about [JCloud here](https://docs.jina.ai/fundamentals/jcloud/).


Expand All @@ -28,9 +24,26 @@ executors:
````

Note that, `port` is unnecessary here as JCloud will assign a new URL for any deployed service.
```{tip}
`port` is unnecessary here as JCloud will assign a new hostname and port for any deployed service.
```

Executors must start with `jinahub+docker://` as it is required by JCloud. We currently provide containerized executors [`jinahub+docker://CLIPTorchEncoder`](https://hub.jina.ai/executor/gzpbl8jh) and [`jinahub+docker://CLIPOnnxEncoder`](https://hub.jina.ai/executor/2a7auwg2) on Jina Hub. They are automatically synced on the new release of `clip_server` module.

To enable GPU on JCloud, you need to configure it in the YAML file and use prebuilt docker GPU images. For example,

```yaml
jtype: Flow
executors:
- uses: jinahub+docker://CLIPTorchEncoder/latest-gpu
jcloud:
resources:
gpu: shared
```

Executors now must start with `jinahub+docker://` as it is required by JCloud. We currently provide containerized executors [`jinahub+docker://CLIPTorchEncoder`](https://hub.jina.ai/executor/gzpbl8jh) and [`jinahub+docker://CLIPOnnxEncoder`](https://hub.jina.ai/executor/2a7auwg2) on Jina Hub. They are automatically synced on the new release of `clip_server` module.
Please refer [here](https://docs.jina.ai/fundamentals/jcloud/yaml-spec/#gpu) for more details on using GPU in JCloud.
Notice that you must specify a docker image GPU tag for your executor to utilize the GPU. For example `latest-gpu`.
See the 'Tag' section in [CLIPTorchEncoder](https://hub.jina.ai/executor/gzpbl8jh) and [CLIPOnnxEncoder](https://hub.jina.ai/executor/2a7auwg2) for docker image GPU tags.

To deploy,

Expand Down

0 comments on commit a5893c7

Please sign in to comment.