Skip to content

Commit

Permalink
docs: add gateway timeout and update shared gpu (#5069)
Browse files Browse the repository at this point in the history
  • Loading branch information
tarrantro committed Aug 17, 2022
1 parent bffe263 commit 840c221
Show file tree
Hide file tree
Showing 2 changed files with 20 additions and 1 deletion.
15 changes: 15 additions & 0 deletions docs/fundamentals/jcloud/advanced.md
Expand Up @@ -103,6 +103,21 @@ executors:
uses: jinahub+docker://Executor1
```

### Timeout

By default, JCloud gateway will close connections that have been idle for over `600` seconds. If you want longer connection timeout threshold, you can consider changing the `timeout` parameter in `gateway`.

```yaml
jtype: Flow
jcloud:
gateway:
ingress: kong
timeout: 600
executors:
- name: executor1
uses: jinahub+docker://Executor1
```

(retention-days)=
## Retention days

Expand Down
6 changes: 5 additions & 1 deletion docs/fundamentals/jcloud/resources.md
Expand Up @@ -95,7 +95,7 @@ When using GPU resources, it may take few extra mins until all Executors ready t

##### Shared

An executor using a `shared` GPU shares this GPU with up to 10 other Executors.
An executor using a `shared` GPU shares this GPU with up to 4 other Executors.
This enables a time-slicing, which allows workloads that land on oversubscribed GPUs to interleave with one another.

```yaml
Expand All @@ -108,6 +108,10 @@ executors:
gpu: shared
```

```{note}
When using shared GPU resources, it will share the GPU memory across pods(24G memory total). If your application is memory consuming, we suggest using a dedicated GPU.
```

```{caution}
There are no special provisions in place to isolate replicas that run on the same underlying GPU. Each workload has access to the GPU memory and runs in the same fault-domain as of all the others. Therefore, if one workload crashes, they all do.
```
Expand Down

0 comments on commit 840c221

Please sign in to comment.