🚀 Dockerize dalai

[bernatvadell]

I've been playing around with this repository and dockerized the app.

During the process I have refactored a few things. Always respecting the original values to avoid breaking-changes.

If you are interested in maintaining support for this feature, I would recommend publishing the image on dockerhub.

I've also included a docker-compose to ease initial startup and a script in the package.json

The only requirement is to have Docker Engine (or Docker Desktop) installed.

and execute yarn run:docker or npm run run:docker

if the image were published in dockerhub it would be as simple as running:

basic:
docker run -p 3000:3000 cocktailpeanut/dalai

specifing model:
docker run -p 3000:3000 -e LLAMA_MODEL=7B cocktailpeanut/dalai

mapping volume models to persist:
docker run -p 3000:3000 -v models:/home/dalai/models cocktailpeanut/dalai

[waffletower]

This is awesome! Will have to try this out!

[adampankow]

Good work on this. It worked great, minus some small issues. The models directory still ended up owned by root, hence not writable. As well, the container needed to be rerun a few times before the model properly converted. Otherwise great though! 👍 👍

[kaminskypavel]

great work @bernatvadell!

I'm using Docker Desktop on windows and the final step was crashing for me.
it seems like ./dalai has some weird newline that gives

/bin/bash^M: bad interpreter: No such file or directory

clearing it with sed -i -e 's/\r$//' ./dalai did the trick.
I suggest you update the file in this PR.

[leeola]

Interesting, this failed to build for me. Error:

Step 23/26 : RUN yarn install
 ---> Running in dda9e9bf1e29
yarn install v1.22.19
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
error Could not write file "/home/dalai/app/yarn-error.log": "EACCES: permission denied, open '/home/dalai/app/yarn-error.log'"
error An unexpected error occurred: "EACCES: permission denied, mkdir '/home/dalai/app/node_modules'".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
The command '/bin/sh -c yarn install' returned a non-zero code: 1

Fixed locally by adding a chown above the workdir, eg:

RUN mkdir /home/dalai/app && chown -R dalai:dalai /home/dalai/app
WORKDIR /home/dalai/app

[valentinvieriu]

Any Reasons you are Hardcoding here the Model 7B?

[bernatvadell]

not hardcoded, you can change during runtime passing env.

docker run -e LLAMA_MODEL=7B ...

by the default we're using 7B (smallest)

[valentinvieriu]

But then you need to add this to the docker-compose.yaml file too. At this moment, if you just say docker-compose up it uses the 7B Model

[bernatvadell]

it is possible that during the last merge, that changed the api something will stop working. I review it and I tell you things.

[valentinvieriu]

Should we add here again command: yarn just:run $LLAMA_MODEL?
When I run docker compose up the env variable in docker-compose.yaml is ignored and the hardcoded one from Dockerfile is used

[evanjrowley]

It would be really nice if the instructions for running this could also work on Fedora, which uses Podman. Compatibility of the container image should not be a problem, but the current directions rely on Docker Compose, which is not well supported by Podman.