sglang 1.0.2 — sm70 patch

Patched power-sglang-cuda124 Python wheel (sm70 / Tesla V100).

The previous 1.0.1 py3-none-any wheel was built without the Volta guard fix (SGLang only supports sm75 and above). This wheel includes model_runner.py allowing sm70.

Install (Python package):

python3 -m pip install https://github.com/RomanMoz/power-sglang-cuda124/releases/download/v1.0.2/power_sglang_cuda124-1.0.2-py3-none-any.whl

Kernel wheel (unchanged, still v1.0.1):

python3 -m pip install https://github.com/RomanMoz/power-sglang-cuda124/releases/download/v1.0.1/power_sgl_kernel_cuda124-1.0.1-cp310-abi3-linux_ppc64le.whl

Serve flags: --attention-backend triton --dtype float16 --tp 2 --mem-fraction-static 0.85 --context-length 8192 --disable-custom-all-reduce

sgl-kernel 1.0.1 — POWER9 / Tesla V100 sm70

Prebuilt power-sgl-kernel-cuda124 for IBM POWER9 (ppc64le) + CUDA 12.4, with Tesla V100 (sm70) gencode.

Hardware: 2× Tesla V100-SXM2 16GB (compute 7.0). FA3/NVFP4/SM100 MXFP8 are off or stubbed.

Install:

python3 -m pip install \
  https://github.com/RomanMoz/power-sglang-cuda124/releases/download/v1.0.1/power_sgl_kernel_cuda124-1.0.1-cp310-abi3-linux_ppc64le.whl

Serve Granite 4.1-8b:

--attention-backend triton --dtype float16 --tp 2 --mem-fraction-static 0.85 --max-model-len 8192

See README.md and GRANITE.md.