Skip to content

v2.5: runs all 5 Pingala-8 models, including Qwen3.5

Latest
Latest

Choose a tag to compare

View all tags
@harshit-genrius harshit-genrius released this 26 Oct 03:07
· 1 commit to main since this release
v2.5
5e03fe2

The one-click build now runs all 5 Pingala-8 models: Qwen3-4B, Llama-3.1-8B, Qwen3.5-4B, Qwen3.5-2B and Qwen3.5-0.8B.

What changed

  • New runtime: llama.cpp 7ac59a6 + the released CUDA patch (patches/llama_cuda_vs_7ac59a6.patch) + patches/q35_loader_count_delta.patch. The loader now checks that the number of coded tensors equals the file's vr.n_linear instead of assuming 7 per layer. The hybrid Qwen3.5-2B and Qwen3.5-0.8B models have 6 in each DeltaNet layer, so earlier one-click builds cannot load them.
  • New launchers, with the same safe download as v2.4.1 (download to <name>.part, check bytes and sha256 from Hugging Face, then rename; 3 tries):
    • Windows: run_qwen35_4b.bat, run_qwen35_2b.bat, run_qwen35_0_8b.bat
    • Linux: ./run.sh qwen35-4b, ./run.sh qwen35-2b, ./run.sh qwen35-0.8b (or run_qwen35_4b.sh, run_qwen35_2b.sh, run_qwen35_0_8b.sh)
  • README.txt lists the 5 models and their download sizes.

What did not change

  • Same build settings as v2.4.1: Windows CUDA 12.8 (MSVC, GitHub Actions, architectures 86;89;120); Linux CUDA 13.2 (architectures 75-virtual;80;86;89;120). Same CUDA libraries, env.bat settings and llama-server command line.
  • The models are unchanged. Model files already in models are kept.
  • The loader change keeps the earlier files working: on the rebuilt runtime Qwen3-4B gives the same WikiText-2 perplexity ([73]16.6413) and token generation at 1.01x of the released build.

Tested (RTX 5070 Ti Laptop GPU): on Linux and on Windows (cmd.exe), each of the 5 models loads in this build and answers a 30-token test prompt. A download cut off mid-file was retried and passed the sha256 check on both systems.

Model Download Windows Linux
Qwen3-4B 1.24 GB run_qwen3_4b.bat ./run.sh qwen
Llama-3.1-8B (base model, Built with Llama) 2.39 GB run_llama31_8b.bat ./run.sh llama
Qwen3.5-4B (text only) 1.51 GB run_qwen35_4b.bat ./run.sh qwen35-4b
Qwen3.5-2B (text only) 0.78 GB run_qwen35_2b.bat ./run.sh qwen35-2b
Qwen3.5-0.8B (text only) 0.35 GB run_qwen35_0_8b.bat ./run.sh qwen35-0.8b

Windows: download pingala8-windows-cuda12.zip, unzip, double-click a launcher. The first run downloads the model from Hugging Face; then a chat page opens at http://127.0.0.1:8080.

Linux: download pingala8-linux-cuda13.tar.gz, extract, run ./run.sh <model> and open http://127.0.0.1:8080.

Needs an NVIDIA GPU (RTX 30, 40 or 50 series) and a recent driver. Stock llama.cpp, Ollama and LM Studio cannot load these files.

Demo video and poster (from v2.4-oneclick):

Paper: https://doi.org/10.5281/zenodo.23481757
Models: https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit, https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit (Built with Llama), https://huggingface.co/Genrius-AI/Qwen3.5-4B-Pingala8-2bit, https://huggingface.co/Genrius-AI/Qwen3.5-2B-Pingala8-2bit, https://huggingface.co/Genrius-AI/Qwen3.5-0.8B-Pingala8-2bit

Assets 4
Loading