Does not compile with new pytorch versions

[styler00dollar]

pip3 install torch==1.10.0+cu113 torchvision==0.11.1+cu113 torchaudio==0.10.0+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html
conda install pytorch=1.10 torchvision torchaudio cudatoolkit=11.3 -c pytorch
conda install pytorch==1.9 torchvision torchaudio cudatoolkit=11.1 -c pytorch -c conda-forge -y

FAILED: /home/Desktop/Mesa/build/temp.linux-x86_64-3.7/native.o 
c++ -MMD -MF /home/Desktop/Mesa/build/temp.linux-x86_64-3.7/native.o.d -pthread -B /home/miniconda3/envs/mesa2/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/miniconda3/envs/mesa2/lib/python3.7/site-packages/torch/include -I/home/miniconda3/envs/mesa2/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/miniconda3/envs/mesa2/lib/python3.7/site-packages/torch/include/TH -I/home/miniconda3/envs/mesa2/lib/python3.7/site-packages/torch/include/THC -I/home/miniconda3/envs/mesa2/include/python3.7m -c -c /home/Desktop/Mesa/native.cpp -o /home/Desktop/Mesa/build/temp.linux-x86_64-3.7/native.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=native -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/home/Desktop/Mesa/native.cpp: In function ‘at::Tensor gelu_backward_cpu(const at::Tensor&, const at::Tensor&)’:
/home/Desktop/Mesa/native.cpp:92:24: error: ‘gelu_backward_cpu’ is not a member of ‘at::native’; did you mean ‘glu_backward_cpu’?
   92 |     return at::native::gelu_backward_cpu(grad_output, input);
      |                        ^~~~~~~~~~~~~~~~~
      |                        glu_backward_cpu
/home/Desktop/Mesa/native.cpp: In function ‘at::Tensor gelu_backward_cuda(const at::Tensor&, const at::Tensor&)’:
/home/Desktop/Mesa/native.cpp:98:24: error: ‘gelu_backward_cuda’ is not a member of ‘at::native’; did you mean ‘glu_backward_cuda’?
   98 |     return at::native::gelu_backward_cuda(grad_output, input);
      |                        ^~~~~~~~~~~~~~~~~~
      |                        glu_backward_cuda
/home/Desktop/Mesa/native.cpp: In function ‘at::Tensor softmax_backward_cpu(const at::Tensor&, const at::Tensor&, int64_t, const at::Tensor&)’:
/home/Desktop/Mesa/native.cpp:190:24: error: ‘softmax_backward_cpu’ is not a member of ‘at::native’; did you mean ‘col2im_backward_cpu’?
  190 |     return at::native::softmax_backward_cpu(grad_output, output, dim, self);
      |                        ^~~~~~~~~~~~~~~~~~~~
      |                        col2im_backward_cpu
/home/Desktop/Mesa/native.cpp: In function ‘at::Tensor softmax_backward_cuda(const at::Tensor&, const at::Tensor&, int64_t, const at::Tensor&)’:
/home/Desktop/Mesa/native.cpp:198:24: error: ‘softmax_backward_cuda’ is not a member of ‘at::native’; did you mean ‘col2im_backward_cuda’?
  198 |     return at::native::softmax_backward_cuda(grad_output, output, dim, self);
      |                        ^~~~~~~~~~~~~~~~~~~~~
      |                        col2im_backward_cuda
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/home/miniconda3/envs/mesa2/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1723, in _run_ninja_build
    env=env)
  File "/home/miniconda3/envs/mesa2/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

These versions seem to work:

conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=10.2 -c pytorch
conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=11.1 -c pytorch -c conda-forge
pip install torch==1.7.1+cu101 torchvision==0.8.2+cu101 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

[HubHop]

Hi @styler00dollar,

Thanks for using Mesa and reporting this issue.

Our experiments reported in the paper are based on PyTorch 1.7 with CUDA 10.1. I have also successfully built Mesa with PyTorch 1.8 and PyTorch 1.9. However, it seems PyTorch 1.10 has changed its C++ API for GELU and Softmax. We will investigate this issue further.

Kind regards.

[HubHop]

Hi @styler00dollar，

This is an update on this issue. We have fixed the compatibility with PyTorch 1.10 and successfully built Mesa with it. Please do not hesitate to contact us if you have any further issues.

Cheers,
Zizheng

[styler00dollar]

Quick testing shows that it does indeed compile on 1.10 and 1.9 now. The above commands work. Thanks.