diff --git a/docs/content/docs/getting-started/build.md b/docs/content/docs/getting-started/build.md index a4db135e652..7e585ab3f68 100644 --- a/docs/content/docs/getting-started/build.md +++ b/docs/content/docs/getting-started/build.md @@ -154,11 +154,11 @@ cd LocalAI # build the binary make build -# Download gpt4all-j to models/ -wget https://gpt4all.io/models/ggml-gpt4all-j.bin -O models/ggml-gpt4all-j +# Download phi-2 to models/ +wget https://huggingface.co/TheBloke/phi-2-GGUF/resolve/main/phi-2.Q2_K.gguf -O models/phi-2.Q2_K # Use a template from the examples -cp -rf prompt-templates/ggml-gpt4all-j.tmpl models/ +cp -rf prompt-templates/ggml-gpt4all-j.tmpl models/phi-2.Q2_K.tmpl # Run LocalAI ./local-ai --models-path=./models/ --debug=true @@ -167,7 +167,7 @@ cp -rf prompt-templates/ggml-gpt4all-j.tmpl models/ curl http://localhost:8080/v1/models curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{ - "model": "ggml-gpt4all-j", + "model": "phi-2.Q2_K", "messages": [{"role": "user", "content": "How are you?"}], "temperature": 0.9 }' @@ -175,9 +175,19 @@ curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/jso #### Troublshooting mac -If you encounter errors regarding a missing utility metal, install `Xcode` from the App Store. -If completions are slow, ensure that `gpu-layers` in your model yaml matches the number of layers from the model in use (or simply use a high number such as 256). -If you a get a compile error: `error: only virtual member functions can be marked 'final'`, reinstall all the necessary brew packages, clean the build, and try again. +1. If you encounter errors regarding a missing utility metal, install `Xcode` from the App Store. +2. After the installation of Xcode, if you receive a xcrun error `'xcrun: error: unable to find utility "metal", not a developer tool or in PATH'`. You might have installed the Xcode command line tools before installing Xcode, the former one is pointing to an incomplete SDK. + +``` +# print /Library/Developer/CommandLineTools, if command line tools were installed in advance +xcode-select --print-path + +# point to a complete SDK +sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer +``` + +3. If completions are slow, ensure that `gpu-layers` in your model yaml matches the number of layers from the model in use (or simply use a high number such as 256). +4. If you a get a compile error: `error: only virtual member functions can be marked 'final'`, reinstall all the necessary brew packages, clean the build, and try again. ``` # reinstall build dependencies