latest

This is a rolling release for branch axllm. Assets in this release are overwritten on each push to axllm.

Fixed download links

All assets can be downloaded via:

• https://github.com/AXERA-TECH/ax-llm/releases/latest/download/<asset_name>

Docker images (offline)

Images are exported as both:

• .tar (no gzip required on target devices)
• .tar.gz (smaller)

Assets:

• axllm-docker-axcl-amd64.tar / axllm-docker-axcl-amd64.tar.gz
• axllm-docker-axcl-arm64.tar / axllm-docker-axcl-arm64.tar.gz
• axllm-docker-ax650-arm64.tar / axllm-docker-ax650-arm64.tar.gz

Load:

docker load -i axllm-docker-axcl-amd64.tar
# or
docker load -i axllm-docker-axcl-amd64.tar.gz

Run (AXCL / PCIe):

• Host must have AXCL driver installed and device nodes present.
• Quick start (simplest): use --privileged.
• If you prefer not to use --privileged, map devices explicitly:
  • --device=/dev/axcl_host
  • --device=/dev/ax_mmb_dev

docker run --rm --network host \
  --device=/dev/axcl_host \
  --device=/dev/ax_mmb_dev \
  -v /path/to/models:/models \
  <loaded-image-tag> serve /models/<model_dir> --port 8000

Run (AX650 board):

docker run --rm --network host --privileged \
  -v /soc:/soc:ro \
  -e LD_LIBRARY_PATH=/soc/lib:$LD_LIBRARY_PATH \
  -v /path/to/models:/models \
  <loaded-image-tag> serve /models/<model_dir> --port 8000

Native build artifacts

• axllm-axcl-linux-amd64 (standalone binary)
• axllm-axcl-linux-arm64 (standalone binary)
• axllm-ax650-linux-arm64 (standalone binary)
• axllm-axcl-linux-riscv64 (standalone binary)
• axllm-axcl-windows-mingw64.zip

Example:

chmod +x axllm-axcl-linux-amd64
./axllm-axcl-linux-amd64 --help