aarch64 linux wheel file does not have image.so

[apivovarov]

🐛 Describe the bug

Start ARM64 Ubuntu 20.04 instance

Install torch and torchvision using pip as recommended at https://pytorch.org/

pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cpu

Import torchvision

import torchvision

Error:

/usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension:
  warn(f"Failed to load image Python extension: {e}")

I checked dist-packages/torchvision folder. Yes, image.so file is missing.

I checked several other wheel files.
Other wheel files have image.so lib

### x86_64 linux - torchvision-0.14.1+cpu-cp38-cp38-linux_x86_64.whl
38,846,032 Dec  9 02:52 _C.so
 9,491,529 Dec  9 02:52 image.so

### aarch64 linux - torchvision-0.14.1-cp38-cp38-manylinux2014_aarch64.whl
38,868,832 Dec 14 22:31 _C.so
                         ### image.so is missing !!!

### arm64 mac - torchvision-0.14.1-cp38-cp38-macosx_11_0_arm64.whl
1,059,888 Dec  9 02:53 _C.so
  213,008 Dec  9 02:53 image.so

Versions

Collecting environment information...
PyTorch version: 1.13.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.5 LTS (aarch64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.31

Python version: 3.8.10 (default, Jun 22 2022, 20:18:18)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.15.0-1026-aws-aarch64-with-glibc2.29
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.5
[pip3] torch==1.13.1
[pip3] torchaudio==0.13.1
[pip3] torchvision==0.14.1
[conda] Could not collect

[apivovarov]

workaround is to build aarch64 linux torchvision wheel from source

sudo apt install libjpeg-turbo8 libjpeg-turbo8-dev \
libpng16-16 libpng-tools libpng-dev

CMAKE_VERSION=3.24.3
wget --quiet https://github.com/Kitware/CMake/releases/download/v${CMAKE_VERSION}/cmake-${CMAKE_VERSION}-linux-aarch64.sh
bash cmake-${CMAKE_VERSION}-linux-aarch64.sh --skip-license --prefix=/usr/local

sudo pip3 install torch --extra-index-url https://download.pytorch.org/whl/cpu
sudo pip3 install ninja

git clone https://github.com/pytorch/vision
cd vision
git checkout v0.14.1

find /usr -name TorchConfig.cmake
# set CMAKE_PREFIX_PATH pointing to parent of Torch/TorchConfig.cmake
export CMAKE_PREFIX_PATH=/usr/local/lib/python3.8/dist-packages/torch/share/cmake

export BUILD_TEST=0

# First, make sure C++ build works fine
mkdir build && cd build
cmake -DUSE_PYTHON=ON ..
make -j$(nproc)
cd ..

# run python wheel build
python3 setup.py build
python3 setup.py bdist_wheel

Output should say

...
building 'torchvision._C' extension
...
building 'torchvision.image' extension
...

Now dist/torchvision-0.14.1a0+5e8e2f1-cp38-cp38-linux_aarch64.whl has image.so lib

38,305,472 Jan 18 00:56 _C.so
 9,300,256 Jan 18 00:57 image.so

[NicolasHug]

Start ARM64 Ubuntu 20.04 instance
Import torchvision

Could you please provide details on how you installed torchvision in the first place? Or is it that ubuntu comes with torchvision pre-installed (in which case this could be a packaging error from Ubuntu) ?

workaround is to build aarch64 linux torchvision wheel from source

Glad it worked, note that you can probably simplify your build process by following / adapting the steps described here https://github.com/pytorch/vision/blob/main/CONTRIBUTING.md#development-installation

[apivovarov]

1. Done - Added torch and torchvision installation step to the description

2. Thank you

3. @NicolasHug Can you confirm that https://download.pytorch.org/whl/torchvision-0.14.1-cp38-cp38-manylinux2014_aarch64.whl does not have image.so? You can simply unzip it and look.

4. I noticed that If particular libs are not installed then python3 setup.py build silently skips building 'torchvision.image' extension step.

[NicolasHug]

Thanks for the details @apivovarov . I can confirm that image.so seems to be missing on he aarch64 wheels (even on 0.13). @malfet @atalman is this something you could look into?

[apivovarov]

Looks like it was fixed in 1.15.2! Thank you!

>>> import torchvision
>>> torchvision.__version__
'0.15.2'

uname -moip
aarch64 aarch64 aarch64 GNU/Linux

$ cd /usr/local/lib/python3.8/dist-packages/torchvision
$ ls -la *.so
-rwxr-xr-x 1 root staff 862688 May 22 21:19 _C.so
-rwxr-xr-x 1 root staff 783888 May 22 21:19 image.so