From ffdf22430f15048d98729cc551f04e6ae56c31f0 Mon Sep 17 00:00:00 2001 From: Jeximo Date: Sat, 4 May 2024 15:00:10 -0300 Subject: [PATCH 1/7] Further tidy on Android instructions README.md Fixed some logic when following readme direction --- ...her tidy on Android instructions README.md | 34 +++++++++---------- 1 file changed, 17 insertions(+), 17 deletions(-) rename README.md => Further tidy on Android instructions README.md (99%) diff --git a/README.md b/Further tidy on Android instructions README.md similarity index 99% rename from README.md rename to Further tidy on Android instructions README.md index 2f1317662cf38..9a8d36ceec8b1 100644 --- a/README.md +++ b/Further tidy on Android instructions README.md @@ -933,8 +933,23 @@ If your issue is with model generation quality, then please at least scan the fo ### Android +#### Build on Android using Termux +[Termux](https://github.com/termux/termux-app#installation) is an alternative to execute `llama.cpp` on an Android device (no root required). +``` +apt update && apt upgrade -y +apt install git +``` + +It's recommended to move your model inside the `~/` directory for best performance: +``` +cd storage/downloads +mv model.gguf ~/ +``` + +[Get the code](https://github.com/ggerganov/llama.cpp#get-the-code) & [follow the Linux build instructions](https://github.com/ggerganov/llama.cpp#build) to build `llama.cpp`. + #### Building the Project using Android NDK -You can easily run `llama.cpp` on Android device with [termux](https://termux.dev/). +You can easily run `llama.cpp` on Android device with [termux](https://github.com/termux/termux-app#installation). First, install the essential packages for termux: ``` @@ -951,7 +966,7 @@ $ export NDK= $ cmake -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod .. $ make ``` -Install [termux](https://termux.dev/) on your device and run `termux-setup-storage` to get access to your SD card. +Install [termux](https://github.com/termux/termux-app#installation) on your device and run `termux-setup-storage` to get access to your SD card (if Android 11+ then run the command twice). Finally, copy these built `llama` binaries and the model file to your device storage. Because the file permissions in the Android sdcard cannot be changed, you can copy the executable files to the `/data/data/com.termux/files/home/bin` path, and then execute the following commands in Termux to add executable permission: (Assumed that you have pushed the built executable files to the /sdcard/llama.cpp/bin path using `adb push`) @@ -977,21 +992,6 @@ Here is a demo of an interactive session running on Pixel 5 phone: https://user-images.githubusercontent.com/271616/225014776-1d567049-ad71-4ef2-b050-55b0b3b9274c.mp4 -#### Build on Android using Termux -[Termux](https://github.com/termux/termux-app#installation) is an alternative to execute `llama.cpp` on an Android device (no root required). -``` -apt update && apt upgrade -y -apt install git -``` - -It's recommended to move your model inside the `~/` directory for best performance: -``` -cd storage/downloads -mv model.gguf ~/ -``` - -[Follow the Linux build instructions](https://github.com/ggerganov/llama.cpp#build) to build `llama.cpp`. - ### Docker #### Prerequisites From 3ce8a9823460b1115f6f9786d6aef57b1651ddc2 Mon Sep 17 00:00:00 2001 From: Jeximo Date: Sun, 5 May 2024 12:17:52 -0300 Subject: [PATCH 2/7] Clean up redundent information A new user arriving will see simple directions on llama.cpp homepage --- ...her tidy on Android instructions README.md | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/Further tidy on Android instructions README.md b/Further tidy on Android instructions README.md index 9a8d36ceec8b1..da91137727c87 100644 --- a/Further tidy on Android instructions README.md +++ b/Further tidy on Android instructions README.md @@ -949,16 +949,9 @@ mv model.gguf ~/ [Get the code](https://github.com/ggerganov/llama.cpp#get-the-code) & [follow the Linux build instructions](https://github.com/ggerganov/llama.cpp#build) to build `llama.cpp`. #### Building the Project using Android NDK -You can easily run `llama.cpp` on Android device with [termux](https://github.com/termux/termux-app#installation). - -First, install the essential packages for termux: -``` -pkg install clang wget git cmake -``` -Second, obtain the [Android NDK](https://developer.android.com/ndk) and then build with CMake: - -You can execute the following commands on your computer to avoid downloading the NDK to your mobile. Of course, you can also do this in Termux. +Obtain the [Android NDK](https://developer.android.com/ndk) and then build with CMake: +Execute the following commands on your computer to avoid downloading the NDK to your mobile. Alternatively, you can also do this in Termux. ``` $ mkdir build-android $ cd build-android @@ -966,7 +959,14 @@ $ export NDK= $ cmake -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod .. $ make ``` + Install [termux](https://github.com/termux/termux-app#installation) on your device and run `termux-setup-storage` to get access to your SD card (if Android 11+ then run the command twice). + +Install the essential packages for termux: +``` +pkg install clang wget git cmake +``` + Finally, copy these built `llama` binaries and the model file to your device storage. Because the file permissions in the Android sdcard cannot be changed, you can copy the executable files to the `/data/data/com.termux/files/home/bin` path, and then execute the following commands in Termux to add executable permission: (Assumed that you have pushed the built executable files to the /sdcard/llama.cpp/bin path using `adb push`) @@ -988,7 +988,7 @@ $cd /data/data/com.termux/files/home/bin $./main -m ../model/llama-2-7b-chat.Q4_K_M.gguf -n 128 -cml ``` -Here is a demo of an interactive session running on Pixel 5 phone: +Here's a demo of an interactive session running on Pixel 5 phone: https://user-images.githubusercontent.com/271616/225014776-1d567049-ad71-4ef2-b050-55b0b3b9274c.mp4 From 34a241be638e53d471786fd0ee5291d572ad835d Mon Sep 17 00:00:00 2001 From: Jeximo Date: Sun, 5 May 2024 12:28:33 -0300 Subject: [PATCH 3/7] corrected puncuation Period after cmake, colon after termux --- Further tidy on Android instructions README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Further tidy on Android instructions README.md b/Further tidy on Android instructions README.md index da91137727c87..463b139c6c44d 100644 --- a/Further tidy on Android instructions README.md +++ b/Further tidy on Android instructions README.md @@ -949,9 +949,9 @@ mv model.gguf ~/ [Get the code](https://github.com/ggerganov/llama.cpp#get-the-code) & [follow the Linux build instructions](https://github.com/ggerganov/llama.cpp#build) to build `llama.cpp`. #### Building the Project using Android NDK -Obtain the [Android NDK](https://developer.android.com/ndk) and then build with CMake: +Obtain the [Android NDK](https://developer.android.com/ndk) and then build with CMake. -Execute the following commands on your computer to avoid downloading the NDK to your mobile. Alternatively, you can also do this in Termux. +Execute the following commands on your computer to avoid downloading the NDK to your mobile. Alternatively, you can also do this in Termux: ``` $ mkdir build-android $ cd build-android From 8ae63953fef3943e22689ab098868a26b78a7acc Mon Sep 17 00:00:00 2001 From: Jeximo Date: Sun, 5 May 2024 18:30:14 -0300 Subject: [PATCH 4/7] re-word for clarity method seems to be more correct, instead of alternative in this context --- Further tidy on Android instructions README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Further tidy on Android instructions README.md b/Further tidy on Android instructions README.md index 463b139c6c44d..c70e4e2473d2c 100644 --- a/Further tidy on Android instructions README.md +++ b/Further tidy on Android instructions README.md @@ -934,7 +934,7 @@ If your issue is with model generation quality, then please at least scan the fo ### Android #### Build on Android using Termux -[Termux](https://github.com/termux/termux-app#installation) is an alternative to execute `llama.cpp` on an Android device (no root required). +[Termux](https://github.com/termux/termux-app#installation) is a method to execute `llama.cpp` on an Android device (no root required). ``` apt update && apt upgrade -y apt install git From 7f0b8dd49528113c324429319e0a4d039220b9e1 Mon Sep 17 00:00:00 2001 From: Jeximo Date: Sun, 5 May 2024 19:47:24 -0300 Subject: [PATCH 5/7] Organized required packages per build type building llama.cpp with NDK on a pc doesn't require installing clang, cmake, git, or wget in termux. --- Further tidy on Android instructions README.md | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/Further tidy on Android instructions README.md b/Further tidy on Android instructions README.md index c70e4e2473d2c..217f50bd19ac1 100644 --- a/Further tidy on Android instructions README.md +++ b/Further tidy on Android instructions README.md @@ -937,7 +937,7 @@ If your issue is with model generation quality, then please at least scan the fo [Termux](https://github.com/termux/termux-app#installation) is a method to execute `llama.cpp` on an Android device (no root required). ``` apt update && apt upgrade -y -apt install git +apt install git make cmake ``` It's recommended to move your model inside the `~/` directory for best performance: @@ -962,11 +962,6 @@ $ make Install [termux](https://github.com/termux/termux-app#installation) on your device and run `termux-setup-storage` to get access to your SD card (if Android 11+ then run the command twice). -Install the essential packages for termux: -``` -pkg install clang wget git cmake -``` - Finally, copy these built `llama` binaries and the model file to your device storage. Because the file permissions in the Android sdcard cannot be changed, you can copy the executable files to the `/data/data/com.termux/files/home/bin` path, and then execute the following commands in Termux to add executable permission: (Assumed that you have pushed the built executable files to the /sdcard/llama.cpp/bin path using `adb push`) From b65d320174010b1ff727b307ad4a0e3ad3851df8 Mon Sep 17 00:00:00 2001 From: Jeximo Date: Mon, 6 May 2024 18:24:56 -0300 Subject: [PATCH 6/7] README.md corrected title --- Further tidy on Android instructions README.md => README.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename Further tidy on Android instructions README.md => README.md (100%) diff --git a/Further tidy on Android instructions README.md b/README.md similarity index 100% rename from Further tidy on Android instructions README.md rename to README.md From 39d928c44452e007c227a52166b83821fe73c711 Mon Sep 17 00:00:00 2001 From: Jeximo Date: Tue, 7 May 2024 11:04:48 -0300 Subject: [PATCH 7/7] fix trailing whitespace --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 217f50bd19ac1..a6768b1540711 100644 --- a/README.md +++ b/README.md @@ -937,7 +937,7 @@ If your issue is with model generation quality, then please at least scan the fo [Termux](https://github.com/termux/termux-app#installation) is a method to execute `llama.cpp` on an Android device (no root required). ``` apt update && apt upgrade -y -apt install git make cmake +apt install git make cmake ``` It's recommended to move your model inside the `~/` directory for best performance: