Skip to content

Commit

Permalink
Python ncnn with vulkan (#5020)
Browse files Browse the repository at this point in the history 
* add option for enabling vulkan in setup.py
* load winograd type options for pybind.
* enable the usage of humbletim/install-vulkan-sdk in ci
* vulkan-loader with release build
* python setup.py install --vulkan=on
* moltenVK for x86_64 macos
* remove universal2  temporarily
  • Loading branch information
@Hideousmon
Hideousmon committed Oct 23, 2023
1 parent 80b3b9c commit a6decad
Show file tree
Hide file tree
Showing 4 changed files with 372 additions and 13 deletions.
271 changes: 261 additions & 10 deletions .github/workflows/release-python.yml
View file
Expand Up @@ -5,6 +5,14 @@ on:
tags:
- '*'

env:
DEVELOPER_DIR: /Applications/Xcode_13.4.1.app/Contents/Developer
MAC_DEPLOYMENT_TARGET: '10.9'
MAC_ARM64_DEPLOYMENT_TARGET: '11.0'
ENABLE_BITCODE: OFF
ENABLE_ARC: OFF
ENABLE_VISIBILITY: OFF

jobs:
build_sdist:
name: Build SDist
Expand Down Expand Up @@ -51,7 +59,6 @@ jobs:
- { os: macos-latest, arch: x86_64, build: 'cp*' }
- { os: macos-latest, arch: x86_64, build: 'pp*' }
- { os: macos-latest, arch: arm64, build: 'cp*' }
- { os: macos-latest, arch: universal2, build: 'cp*' }

steps:
- uses: actions/checkout@v4
Expand All @@ -62,20 +69,223 @@ jobs:
with:
python-version: '3.x'

- name: brew uninstall libomp
# build wheels for ubuntu-20.04
- name: Build wheels for ubuntu manylinux
if: matrix.os == 'ubuntu-20.04' && matrix.build != 'cp*-musllinux*'
uses: pypa/cibuildwheel@v2.16.2
env:
CIBW_ARCHS_LINUX: ${{ matrix.arch }}
CIBW_BUILD: ${{ matrix.build }}
CIBW_BUILD_VERBOSITY: 1
CIBW_ENVIRONMENT: CMAKE_BUILD_PARALLEL_LEVEL=2 NCNN_VULKAN=ON
VULKAN_SDK=/project/Vulkan-Loader/build/Vulkan-Headers
LD_LIBRARY_PATH=/project/Vulkan-Loader/build/loader
CIBW_BEFORE_ALL: yum -y install libXrandr-devel &&
git clone https://github.com/KhronosGroup/Vulkan-Loader.git &&
cd Vulkan-Loader && mkdir build && cd build &&
../scripts/update_deps.py &&
cmake -DCMAKE_BUILD_TYPE=Release -DVULKAN_HEADERS_INSTALL_DIR=$(pwd)/Vulkan-Headers/build/install .. &&
make -j$(nproc) &&
cd Vulkan-Headers &&
ln -s ../loader lib
with:
output-dir: wheelhouse

- name: Build wheels for ubuntu musllinux
if: matrix.os == 'ubuntu-20.04' && matrix.build == 'cp*-musllinux*'
uses: pypa/cibuildwheel@v2.16.2
env:
CIBW_ARCHS_LINUX: ${{ matrix.arch }}
CIBW_BUILD: ${{ matrix.build }}
CIBW_BUILD_VERBOSITY: 1
CIBW_ENVIRONMENT: CMAKE_BUILD_PARALLEL_LEVEL=2 NCNN_VULKAN=ON
VULKAN_SDK=/project/Vulkan-Loader/build/Vulkan-Headers
LD_LIBRARY_PATH=/project/Vulkan-Loader/build/loader
CIBW_BEFORE_ALL: apk add libxrandr-dev &&
git clone https://github.com/KhronosGroup/Vulkan-Loader.git &&
cd Vulkan-Loader && mkdir build && cd build &&
../scripts/update_deps.py &&
cmake -DCMAKE_BUILD_TYPE=Release -DVULKAN_HEADERS_INSTALL_DIR=$(pwd)/Vulkan-Headers/build/install .. &&
make -j$(nproc) &&
cd Vulkan-Headers &&
ln -s ../loader lib
with:
output-dir: wheelhouse

# build wheels for windows-2019
- name: Build wheels for windows amd64
if: matrix.os == 'windows-2019' && matrix.arch == 'AMD64'
uses: pypa/cibuildwheel@v2.16.2
env:
CIBW_ARCHS_WINDOWS: ${{ matrix.arch }}
CIBW_BUILD: ${{ matrix.build }}
CIBW_BUILD_VERBOSITY: 1
CIBW_ENVIRONMENT_WINDOWS: >
PATH="D:\\a\\ncnn\\ncnn\\Vulkan-Loader\\build\\loader\\Release;$PATH"
CMAKE_BUILD_PARALLEL_LEVEL=2 NCNN_VULKAN=ON
VULKAN_SDK=D:/a/ncnn/ncnn/Vulkan-Loader/external/Vulkan-Headers
CIBW_BEFORE_ALL: git clone https://github.com/KhronosGroup/Vulkan-Loader.git &&
cd Vulkan-Loader && mkdir build && cd build &&
python3 ../scripts/update_deps.py --dir ../external --config release &&
cmake -C ../external/helper.cmake -G "Visual Studio 16 2019" -A x64 -DCMAKE_BUILD_TYPE=Release .. &&
cmake --build . --config Release &&
mklink /d "D:/a/ncnn/ncnn/Vulkan-Loader/external/Vulkan-Headers/build/install/lib"
"D:/a/ncnn/ncnn/Vulkan-Loader/build/loader/Release"
CIBW_BEFORE_BUILD: pip install delvewheel
CIBW_REPAIR_WHEEL_COMMAND: delvewheel repair -w {dest_dir} {wheel}
with:
output-dir: wheelhouse

- name: Build wheels for windows x86
if: matrix.os == 'windows-2019' && matrix.arch == 'x86'
uses: pypa/cibuildwheel@v2.16.2
env:
CIBW_ARCHS_WINDOWS: ${{ matrix.arch }}
CIBW_BUILD: ${{ matrix.build }}
CIBW_BUILD_VERBOSITY: 1
CIBW_ENVIRONMENT_WINDOWS: >
PATH="D:\\a\\ncnn\\ncnn\\Vulkan-Loader\\build\\loader\\Release;$PATH"
CMAKE_BUILD_PARALLEL_LEVEL=2 NCNN_VULKAN=ON
VULKAN_SDK=D:/a/ncnn/ncnn/Vulkan-Loader/external/Vulkan-Headers
CIBW_BEFORE_ALL: git clone https://github.com/KhronosGroup/Vulkan-Loader.git &&
cd Vulkan-Loader && mkdir build && cd build &&
python3 ../scripts/update_deps.py --dir ../external --arch ${{ matrix.arch }} --config release &&
cmake -C ../external/helper.cmake -G "Visual Studio 16 2019" -A Win32 -DCMAKE_BUILD_TYPE=Release .. &&
cmake --build . --config Release &&
mklink /d "D:/a/ncnn/ncnn/Vulkan-Loader/external/Vulkan-Headers/build/install/lib"
"D:/a/ncnn/ncnn/Vulkan-Loader/build/loader/Release"
CIBW_BEFORE_BUILD: pip install delvewheel
CIBW_REPAIR_WHEEL_COMMAND: delvewheel repair -w {dest_dir} {wheel}
with:
output-dir: wheelhouse

- name: Build wheels for windows ARM64
if: matrix.os == 'windows-2019' && matrix.arch == 'ARM64'
uses: pypa/cibuildwheel@v2.16.2
env:
CIBW_ARCHS_WINDOWS: ${{ matrix.arch }}
CIBW_BUILD: ${{ matrix.build }}
CIBW_BUILD_VERBOSITY: 1
CIBW_ENVIRONMENT_WINDOWS: >
PATH="D:\\a\\ncnn\\ncnn\\Vulkan-Loader\\build\\loader\\Release;$PATH"
CMAKE_BUILD_PARALLEL_LEVEL=2 NCNN_VULKAN=ON
VULKAN_SDK=D:/a/ncnn/ncnn/Vulkan-Loader/external/Vulkan-Headers
CIBW_BEFORE_ALL: git clone https://github.com/KhronosGroup/Vulkan-Loader.git &&
cd Vulkan-Loader && mkdir build && cd build &&
python3 ../scripts/update_deps.py --dir ../external --config release &&
cmake -C ../external/helper.cmake -G "Visual Studio 16 2019" -A ARM64 -DCMAKE_BUILD_TYPE=Release -DUSE_MASM=OFF .. &&
cmake --build . --config Release &&
mklink /d "D:/a/ncnn/ncnn/Vulkan-Loader/external/Vulkan-Headers/build/install/lib"
"D:/a/ncnn/ncnn/Vulkan-Loader/build/loader/Release"
CIBW_BEFORE_BUILD: pip install delvewheel
CIBW_REPAIR_WHEEL_COMMAND: delvewheel repair -w {dest_dir} {wheel} --no-dll "msvcp140.dll;vcomp140.dll"
with:
output-dir: wheelhouse

# build wheels for macos-latest
- name: cache-openmp for macos
if: matrix.os == 'macos-latest'
id: cache-openmp
uses: actions/cache@v3
with:
path: openmp-install
key: openmp-macos-install-20230504

- name: openmp for macos
if: matrix.os == 'macos-latest' && steps.cache-openmp.outputs.cache-hit != 'true'
run: |
brew uninstall --ignore-dependencies libomp
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-11.0.0/openmp-11.0.0.src.tar.xz
tar -xf openmp-11.0.0.src.tar.xz
cd openmp-11.0.0.src
sed -i'' -e '/.size __kmp_unnamed_critical_addr/d' runtime/src/z_Linux_asm.S
sed -i'' -e 's/__kmp_unnamed_critical_addr/___kmp_unnamed_critical_addr/g' runtime/src/z_Linux_asm.S
- name: Build wheels
- name: openmp-build-x86_64 for macos
if: matrix.os == 'macos-latest' && steps.cache-openmp.outputs.cache-hit != 'true'
run: |
cd openmp-11.0.0.src
mkdir -p build-x86_64 && cd build-x86_64
cmake -DCMAKE_TOOLCHAIN_FILE=$GITHUB_WORKSPACE/toolchains/ios.toolchain.cmake -DPLATFORM=MAC -DARCHS="x86_64" \
-DDEPLOYMENT_TARGET=$MAC_DEPLOYMENT_TARGET -DENABLE_BITCODE=$ENABLE_BITCODE -DENABLE_ARC=$ENABLE_ARC -DENABLE_VISIBILITY=$ENABLE_VISIBILITY \
-DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=install \
-DPERL_EXECUTABLE=/usr/local/bin/perl \
-DLIBOMP_ENABLE_SHARED=OFF -DLIBOMP_OMPT_SUPPORT=OFF -DLIBOMP_USE_HWLOC=OFF ..
cmake --build . -j 3
cmake --build . --target install
- name: openmp-build-arm64 for macos
if: matrix.os == 'macos-latest' && steps.cache-openmp.outputs.cache-hit != 'true'
run: |
cd openmp-11.0.0.src
mkdir -p build-arm64 && cd build-arm64
cmake -DCMAKE_TOOLCHAIN_FILE=$GITHUB_WORKSPACE/toolchains/ios.toolchain.cmake -DPLATFORM=MAC_ARM64 -DARCHS="arm64" \
-DDEPLOYMENT_TARGET=$MAC_ARM64_DEPLOYMENT_TARGET -DENABLE_BITCODE=$ENABLE_BITCODE -DENABLE_ARC=$ENABLE_ARC -DENABLE_VISIBILITY=$ENABLE_VISIBILITY \
-DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=install \
-DPERL_EXECUTABLE=/usr/local/bin/perl \
-DLIBOMP_ENABLE_SHARED=OFF -DLIBOMP_OMPT_SUPPORT=OFF -DLIBOMP_USE_HWLOC=OFF ..
cmake --build . -j 3
cmake --build . --target install
- name: openmp-merge-fat-library for macos
if: matrix.os == 'macos-latest' && steps.cache-openmp.outputs.cache-hit != 'true'
run: |
mkdir -p $GITHUB_WORKSPACE/openmp-install
cp -a openmp-11.0.0.src/build-x86_64/install/include $GITHUB_WORKSPACE/openmp-install
mkdir -p $GITHUB_WORKSPACE/openmp-install/lib
lipo -create \
openmp-11.0.0.src/build-x86_64/install/lib/libomp.a \
openmp-11.0.0.src/build-arm64/install/lib/libomp.a \
-o $GITHUB_WORKSPACE/openmp-install/lib/libomp.a
- name: install-openmp for macos
if: matrix.os == 'macos-latest'
run: |
sudo cp $GITHUB_WORKSPACE/openmp-install/include/* $DEVELOPER_DIR/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
sudo cp $GITHUB_WORKSPACE/openmp-install/lib/libomp.a $DEVELOPER_DIR/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib
- name: vulkansdk for macos
if: matrix.os == 'macos-latest'
run: |
wget https://sdk.lunarg.com/sdk/download/1.3.236.0/mac/vulkansdk-macos-1.3.236.0.dmg?Human=true -O vulkansdk-macos-1.3.236.0.dmg
hdiutil attach vulkansdk-macos-1.3.236.0.dmg
sudo /Volumes/vulkansdk-macos-1.3.236.0/InstallVulkan.app/Contents/MacOS/InstallVulkan --root $GITHUB_WORKSPACE/vulkansdk-macos-1.3.236.0 --accept-licenses --default-answer --confirm-command install
hdiutil detach /Volumes/vulkansdk-macos-1.3.236.0
- name: Build wheels for macos x86_64
if: matrix.os == 'macos-latest' && matrix.arch == 'x86_64'
uses: pypa/cibuildwheel@v2.16.2
env:
CIBW_ARCHS_MACOS: ${{ matrix.arch }}
CIBW_ARCHS_LINUX: ${{ matrix.arch }}
CIBW_ARCHS_WINDOWS: ${{ matrix.arch }}
CIBW_BUILD: ${{ matrix.build }}
CIBW_BUILD_VERBOSITY: 1
CIBW_ENVIRONMENT: CMAKE_BUILD_PARALLEL_LEVEL=2
CIBW_ENVIRONMENT: CMAKE_BUILD_PARALLEL_LEVEL=2 NCNN_VULKAN=ON VULKAN_SDK=$GITHUB_WORKSPACE/vulkansdk-macos-1.3.236.0/macOS
CMAKE_TOOLCHAIN_FILE=$GITHUB_WORKSPACE/toolchains/ios.toolchain.cmake PLATFORM=MAC ARCHS="x86_64"
DEPLOYMENT_TARGET="10.9" ENABLE_BITCODE=OFF ENABLE_ARC=OFF ENABLE_VISIBILITY=OFF
OpenMP_C_FLAGS="-Xclang -fopenmp" OpenMP_CXX_FLAGS="-Xclang -fopenmp"
OpenMP_C_LIB_NAMES="libomp" OpenMP_CXX_LIB_NAMES="libomp"
OpenMP_libomp_LIBRARY="$DEVELOPER_DIR/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib/libomp.a"
Vulkan_INCLUDE_DIR=$GITHUB_WORKSPACE/vulkansdk-macos-1.3.236.0/MoltenVK/include
Vulkan_LIBRARY=$GITHUB_WORKSPACE/vulkansdk-macos-1.3.236.0/MoltenVK/dylib/macOS/libMoltenVK.dylib
with:
output-dir: wheelhouse

- name: Build wheels for macos arm64
if: matrix.os == 'macos-latest' && matrix.arch == 'arm64'
uses: pypa/cibuildwheel@v2.16.2
env:
CIBW_ARCHS_MACOS: ${{ matrix.arch }}
CIBW_BUILD: ${{ matrix.build }}
CIBW_BUILD_VERBOSITY: 1
CIBW_ENVIRONMENT: CMAKE_BUILD_PARALLEL_LEVEL=2 NCNN_VULKAN=ON VULKAN_SDK=$GITHUB_WORKSPACE/vulkansdk-macos-1.3.236.0/macOS
CMAKE_TOOLCHAIN_FILE=$GITHUB_WORKSPACE/toolchains/ios.toolchain.cmake PLATFORM=MAC_ARM64 ARCHS="arm64"
DEPLOYMENT_TARGET="11.0" ENABLE_BITCODE=OFF ENABLE_ARC=OFF ENABLE_VISIBILITY=OFF
OpenMP_C_FLAGS="-Xclang -fopenmp" OpenMP_CXX_FLAGS="-Xclang -fopenmp"
OpenMP_C_LIB_NAMES="libomp" OpenMP_CXX_LIB_NAMES="libomp"
OpenMP_libomp_LIBRARY="$DEVELOPER_DIR/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib/libomp.a"
Vulkan_INCLUDE_DIR=$GITHUB_WORKSPACE/vulkansdk-macos-1.3.236.0/MoltenVK/include
Vulkan_LIBRARY=$GITHUB_WORKSPACE/vulkansdk-macos-1.3.236.0/MoltenVK/dylib/macOS/libMoltenVK.dylib
with:
output-dir: wheelhouse

- name: Show files
run: ls -lh wheelhouse
Expand All @@ -98,7 +308,11 @@ jobs:
fail-fast: false
matrix:
arch: [aarch64, ppc64le, s390x]
build: ['cp36-*', 'cp37-*', 'cp38-*', 'cp39-*', 'cp310-*', 'cp311-*', 'cp312-*']
build: [ 'cp36-manylinux*', 'cp37-manylinux*', 'cp38-manylinux*',
'cp39-manylinux*', 'cp310-manylinux*', 'cp311-manylinux*',
'cp312-manylinux*', 'cp36-musllinux*', 'cp37-musllinux*',
'cp38-musllinux*', 'cp39-musllinux*', 'cp310-musllinux*',
'cp311-musllinux*', 'cp312-musllinux*' ]
include:
- arch: aarch64
build: 'pp37-*'
Expand All @@ -123,13 +337,50 @@ jobs:
with:
platforms: all

- name: Build wheels
- name: Build wheels for manylinux with qemu
if: (matrix.build != 'cp36-musllinux*') && (matrix.build != 'cp37-musllinux*') &&
(matrix.build != 'cp38-musllinux*') && (matrix.build != 'cp39-musllinux*') &&
(matrix.build != 'cp310-musllinux*') && (matrix.build != 'cp311-musllinux*') &&
(matrix.build != 'cp312-musllinux*')
uses: pypa/cibuildwheel@v2.16.2
env:
CIBW_ARCHS_LINUX: ${{ matrix.arch }}
CIBW_BUILD: ${{ matrix.build }}
CIBW_BUILD_VERBOSITY: 1
CIBW_ENVIRONMENT: CMAKE_BUILD_PARALLEL_LEVEL=2
CIBW_ENVIRONMENT: CMAKE_BUILD_PARALLEL_LEVEL=2 NCNN_VULKAN=ON VULKAN_SDK=/project/Vulkan-Loader/build/Vulkan-Headers
LD_LIBRARY_PATH=/project/Vulkan-Loader/build/loader
CIBW_BEFORE_ALL: yum -y install libXrandr-devel &&
git clone https://github.com/KhronosGroup/Vulkan-Loader.git &&
cd Vulkan-Loader && mkdir build && cd build &&
../scripts/update_deps.py &&
cmake -DCMAKE_BUILD_TYPE=Release -DVULKAN_HEADERS_INSTALL_DIR=$(pwd)/Vulkan-Headers/build/install .. &&
make -j$(nproc) &&
cd Vulkan-Headers &&
ln -s ../loader lib
with:
output-dir: wheelhouse

- name: Build wheels for musllinux with qemu
if: (matrix.build == 'cp36-musllinux*') || (matrix.build == 'cp37-musllinux*') ||
(matrix.build == 'cp38-musllinux*') || (matrix.build == 'cp39-musllinux*') ||
(matrix.build == 'cp310-musllinux*') || (matrix.build == 'cp311-musllinux*') ||
(matrix.build == 'cp312-musllinux*')
uses: pypa/cibuildwheel@v2.16.2
env:
CIBW_ARCHS_LINUX: ${{ matrix.arch }}
CIBW_BUILD: ${{ matrix.build }}
CIBW_BUILD_VERBOSITY: 1
CIBW_ENVIRONMENT: CMAKE_BUILD_PARALLEL_LEVEL=2 NCNN_VULKAN=ON VULKAN_SDK=/project/Vulkan-Loader/build/Vulkan-Headers LD_LIBRARY_PATH=/project/Vulkan-Loader/build/loader
CIBW_BEFORE_ALL: apk add libxrandr-dev &&
git clone https://github.com/KhronosGroup/Vulkan-Loader.git &&
cd Vulkan-Loader && mkdir build && cd build &&
../scripts/update_deps.py &&
cmake -DCMAKE_BUILD_TYPE=Release -DVULKAN_HEADERS_INSTALL_DIR=$(pwd)/Vulkan-Headers/build/install .. &&
make -j$(nproc) &&
cd Vulkan-Headers &&
ln -s ../loader lib
with:
output-dir: wheelhouse

- name: Show files
run: ls -lh wheelhouse
Expand Down
40 changes: 38 additions & 2 deletions python/README.md
View file
Expand Up @@ -33,7 +33,41 @@ If you want to build ncnn with some options not as default, or just like to buil
* Visual Studio 2015 or higher
* CMake >= 3.4

## Build
## Build & Install

1. clone ncnn and init submodule.

```bash
cd /pathto/ncnn
git submodule init && git submodule update
```

2. build and install.

```
python setup.py install
```

if you want to enable the usage of vulkan, you can install as following:

```
python setup.py install --vulkan=on
```

> **Attention:**
>
> To enable Vulkan support, you must first install the Vulkan SDK.
>
> **For Windows or Linux Users:**
>
> Ensure that the `VULKAN_SDK` environment variable is set to the path of the Vulkan SDK.
>
> **For MacOS Users:**
>
> On MacOS, you will need to specify additional environment variables. For guidance on setting these variables, please refer to lines 279-286 in the following file: [ncnn/.github/workflows/release-python.yml at master · Tencent/ncnn](https://github.com/Tencent/ncnn/blob/master/.github/workflows/release-python.yml).
## Custom-build & Install

1. clone ncnn and init submodule.
```bash
cd /pathto/ncnn
Expand All @@ -47,7 +81,8 @@ cmake -DNCNN_PYTHON=ON ..
make
```

## Install
3. install

```bash
cd /pathto/ncnn
pip install .
Expand All @@ -60,6 +95,7 @@ python3 setup.py install
```

## Tests

**test**
```bash
cd /pathto/ncnn/python
Expand Down
3 changes: 3 additions & 0 deletions python/src/main.cpp
View file
Expand Up @@ -185,6 +185,9 @@ PYBIND11_MODULE(ncnn, m)
#endif // NCNN_VULKAN
.def_readwrite("openmp_blocktime", &Option::openmp_blocktime)
.def_readwrite("use_winograd_convolution", &Option::use_winograd_convolution)
.def_readwrite("use_winograd23_convolution", &Option::use_winograd23_convolution)
.def_readwrite("use_winograd43_convolution", &Option::use_winograd43_convolution)
.def_readwrite("use_winograd63_convolution", &Option::use_winograd63_convolution)
.def_readwrite("use_sgemm_convolution", &Option::use_sgemm_convolution)
.def_readwrite("use_int8_inference", &Option::use_int8_inference)
.def_readwrite("use_vulkan_compute", &Option::use_vulkan_compute)
Expand Down

0 comments on commit a6decad

Please sign in to comment.