Skip to content

Commit

Permalink
Update inference installation (#1550)
Browse files Browse the repository at this point in the history
* update docs with requirements

* pin fastapi version to avoid middleware error

* update name

* add missing postgres container

* add api key for debugging
  • Loading branch information
jackapbutler committed Feb 13, 2023
1 parent 00fcd54 commit 9216381
Show file tree
Hide file tree
Showing 2 changed files with 28 additions and 7 deletions.
33 changes: 27 additions & 6 deletions inference/README.md
Expand Up @@ -52,14 +52,35 @@ docker attach open-assistant-inference-text-client-1
> **Note:** Please wait for the `inference-text-generation-server` service to
> output `{"message":"Connected"}` before starting to chat.
## Development Variant 2 (you'll need tmux)
## Development Variant 2 (tmux terminal multiplexing)

Run `./full-dev-setup.sh` to start the full development setup. Make sure to wait
until the 2nd terminal is ready and says `{"message":"Connected"}` before
entering input into the last terminal.
Ensure you have `tmux` installed on you machine and the following packages
installed into the Python environment;

- `uvicorn`
- `worker/requirements.txt`
- `server/requirements.txt`
- `text-client/requirements.txt`
- `oasst_shared`

You can run development setup to start the full development setup.

```bash
cd inference
./full-dev-setup.sh
```

> Make sure to wait until the 2nd terminal is ready and says
> `{"message":"Connected"}` before entering input into the last terminal.
## Development Variant 3 (you'll need multiple terminals)

Run a postgres container:

```bash
docker run --rm -it -p 5432:5432 -e POSTGRES_PASSWORD=postgres --name postgres postgres
```

Run a redis container (or use the one of the general docker compose file):

```bash
Expand All @@ -71,7 +92,7 @@ Run the inference server:
```bash
cd server
pip install -r requirements.txt
uvicorn main:app --reload
DEBUG_API_KEYS='["0000"]' uvicorn main:app --reload
```

Run one (or more) workers:
Expand All @@ -89,7 +110,7 @@ running:
docker run --rm -it -p 8001:80 -e MODEL_ID=distilgpt2 ghcr.io/huggingface/text-generation-inference
```

Run the client:
Run the text client:

```bash
cd text-client
Expand Down
2 changes: 1 addition & 1 deletion inference/server/requirements.txt
@@ -1,5 +1,5 @@
alembic
fastapi[all]
fastapi[all]==0.88.0
loguru
prometheus-fastapi-instrumentator
psycopg2-binary
Expand Down

0 comments on commit 9216381

Please sign in to comment.