llama.cpp - Vulkan Hardware Accelerated Build (Windows x64)

Description
This release provides a pre-compiled version of llama.cpp with Vulkan support enabled. This allows for high-performance LLM inference on a variety of GPUs, including AMD, Intel, and NVIDIA.

I have included a pre-configured environment to help you get started immediately.

What's inside
llama-vulkan-windows.zip: Contains the compiled binaries (including llama-server.exe) and necessary dependencies.

Run Script: A .bat script is provided to simplify the launch process.

How to use
Download and extract llama-vulkan-windows.zip.

Place your favorite .gguf model in the folder (or update the path in the script).

Run the provided batch script. It is configured as follows:

Extrait de code
"%LLAMA%\llama-server.exe" ^
-m "%MODEL%" ^
%GPU_ARGS% ^
-c 4096 ^
-t 8 ^
--host 0.0.0.0 ^
--port %PORT%

Access the web interface at http://localhost:8080 (or your chosen port).

Key Parameters in this Build
Vulkan Acceleration: Use %GPU_ARGS% (e.g., -ngl 33) to offload layers to your GPU.

Context Window: Set to 4096 by default for a good balance between memory and performance.

Threads: Set to 8 to maximize CPU efficiency during prompt processing.

Server Mode: Runs as a local API/Web server accessible from your local network.

Requirements
GPU Drivers: Latest drivers for your AMD, Intel, or NVIDIA card.

Vulkan Runtime: Ensure Vulkan is supported by your hardware.

Visual C++ Redistributable: Recommended for Windows users.

Note: This build is intended for testing and local deployment. Based on the excellent work by the llama.cpp community.