Skip to content

Commit

Permalink
Issue-1720: Updated Build on mac documentations (#2171)
Browse files Browse the repository at this point in the history
updated build on macs documentation

Signed-off-by: LeonSijiaLu <leonsijialu1@gmail.com>
  • Loading branch information
LeonSijiaLu committed Apr 28, 2024
1 parent a78cd67 commit 26e1892
Showing 1 changed file with 17 additions and 7 deletions.
24 changes: 17 additions & 7 deletions docs/content/docs/getting-started/build.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,11 +154,11 @@ cd LocalAI
# build the binary
make build
# Download gpt4all-j to models/
wget https://gpt4all.io/models/ggml-gpt4all-j.bin -O models/ggml-gpt4all-j
# Download phi-2 to models/
wget https://huggingface.co/TheBloke/phi-2-GGUF/resolve/main/phi-2.Q2_K.gguf -O models/phi-2.Q2_K
# Use a template from the examples
cp -rf prompt-templates/ggml-gpt4all-j.tmpl models/
cp -rf prompt-templates/ggml-gpt4all-j.tmpl models/phi-2.Q2_K.tmpl
# Run LocalAI
./local-ai --models-path=./models/ --debug=true
Expand All @@ -167,17 +167,27 @@ cp -rf prompt-templates/ggml-gpt4all-j.tmpl models/
curl http://localhost:8080/v1/models
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "ggml-gpt4all-j",
"model": "phi-2.Q2_K",
"messages": [{"role": "user", "content": "How are you?"}],
"temperature": 0.9
}'
```

#### Troublshooting mac

If you encounter errors regarding a missing utility metal, install `Xcode` from the App Store.
If completions are slow, ensure that `gpu-layers` in your model yaml matches the number of layers from the model in use (or simply use a high number such as 256).
If you a get a compile error: `error: only virtual member functions can be marked 'final'`, reinstall all the necessary brew packages, clean the build, and try again.
1. If you encounter errors regarding a missing utility metal, install `Xcode` from the App Store.
2. After the installation of Xcode, if you receive a xcrun error `'xcrun: error: unable to find utility "metal", not a developer tool or in PATH'`. You might have installed the Xcode command line tools before installing Xcode, the former one is pointing to an incomplete SDK.

```
# print /Library/Developer/CommandLineTools, if command line tools were installed in advance
xcode-select --print-path
# point to a complete SDK
sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer
```

3. If completions are slow, ensure that `gpu-layers` in your model yaml matches the number of layers from the model in use (or simply use a high number such as 256).
4. If you a get a compile error: `error: only virtual member functions can be marked 'final'`, reinstall all the necessary brew packages, clean the build, and try again.

```
# reinstall build dependencies
Expand Down

0 comments on commit 26e1892

Please sign in to comment.