Patched power-sglang-cuda124 Python wheel (sm70 / Tesla V100).
The previous 1.0.1 py3-none-any wheel was built without the Volta guard fix (SGLang only supports sm75 and above). This wheel includes model_runner.py allowing sm70.
Install (Python package):
python3 -m pip install https://github.com/RomanMoz/power-sglang-cuda124/releases/download/v1.0.2/power_sglang_cuda124-1.0.2-py3-none-any.whl
Kernel wheel (unchanged, still v1.0.1):
python3 -m pip install https://github.com/RomanMoz/power-sglang-cuda124/releases/download/v1.0.1/power_sgl_kernel_cuda124-1.0.1-cp310-abi3-linux_ppc64le.whl
Serve flags: --attention-backend triton --dtype float16 --tp 2 --mem-fraction-static 0.85 --context-length 8192 --disable-custom-all-reduce