Apex does not build in example docker image

[sidnarayanan]

Trying to build the example image throws the following error. I find the same behavior when trying to build against pytorch 1.2.0/CUDA 10.0 (pytorch installed from a wheel). I can compile against pytorch 1.4.0/CUDA 10.0.

To build the example docker image:

$ wget https://raw.githubusercontent.com/NVIDIA/apex/master/examples/docker/Dockerfile
$ docker build .

Throws:

[...]
    csrc/mlp.cpp:123:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
       AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
       ^
    csrc/mlp.cpp:141:43: error: expected primary-expression before ‘>’ token
             fprop_outputs[0].data_ptr<scalar_t>(),
                                               ^
    /opt/conda/lib/python3.6/site-packages/torch/include/ATen/Dispatch.h:12:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
         return __VA_ARGS__();                          \
                ^~~~~~~~~~~
    csrc/mlp.cpp:123:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
       AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
       ^
    csrc/mlp.cpp:141:45: error: expected primary-expression before ‘)’ token
             fprop_outputs[0].data_ptr<scalar_t>(),
                                                 ^
    /opt/conda/lib/python3.6/site-packages/torch/include/ATen/Dispatch.h:12:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
         return __VA_ARGS__();                          \
                ^~~~~~~~~~~
    csrc/mlp.cpp:123:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
       AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
       ^
    csrc/mlp.cpp:147:46: error: expected primary-expression before ‘>’ token
             grad_o.contiguous().data_ptr<scalar_t>(),
                                                  ^
    /opt/conda/lib/python3.6/site-packages/torch/include/ATen/Dispatch.h:12:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
         return __VA_ARGS__();                          \
                ^~~~~~~~~~~
    csrc/mlp.cpp:123:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
       AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
       ^
    csrc/mlp.cpp:147:48: error: expected primary-expression before ‘)’ token
             grad_o.contiguous().data_ptr<scalar_t>(),
                                                    ^
    /opt/conda/lib/python3.6/site-packages/torch/include/ATen/Dispatch.h:12:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
         return __VA_ARGS__();                          \
                ^~~~~~~~~~~
    csrc/mlp.cpp:123:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
       AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
       ^
    csrc/mlp.cpp:148:43: error: expected primary-expression before ‘>’ token
             fprop_outputs[1].data_ptr<scalar_t>(),
                                               ^
    /opt/conda/lib/python3.6/site-packages/torch/include/ATen/Dispatch.h:12:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
         return __VA_ARGS__();                          \
                ^~~~~~~~~~~
    csrc/mlp.cpp:123:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
       AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
       ^
    csrc/mlp.cpp:148:45: error: expected primary-expression before ‘)’ token
             fprop_outputs[1].data_ptr<scalar_t>(),
                                                 ^
    /opt/conda/lib/python3.6/site-packages/torch/include/ATen/Dispatch.h:12:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
         return __VA_ARGS__();                          \
                ^~~~~~~~~~~
    csrc/mlp.cpp:123:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
       AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
       ^
    csrc/mlp.cpp:149:37: error: expected primary-expression before ‘>’ token
             work_space.data_ptr<scalar_t>(),
                                         ^
    /opt/conda/lib/python3.6/site-packages/torch/include/ATen/Dispatch.h:12:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
         return __VA_ARGS__();                          \
                ^~~~~~~~~~~
    csrc/mlp.cpp:123:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
       AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
       ^
    csrc/mlp.cpp:149:39: error: expected primary-expression before ‘)’ token
             work_space.data_ptr<scalar_t>(),
                                           ^
    /opt/conda/lib/python3.6/site-packages/torch/include/ATen/Dispatch.h:12:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
         return __VA_ARGS__();                          \
                ^~~~~~~~~~~
    csrc/mlp.cpp:123:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES_AND_HALF’
       AT_DISPATCH_FLOATING_TYPES_AND_HALF(inputs[0].type(), "mlp_backward", [&] {
       ^
    error: command 'gcc' failed with exit status 1
  Running setup.py install for apex: finished with status 'error'

[...]

Can you clarify (a) if this is expected behavior and (b) if so, exactly which pytorch/CUDA versions are supported? The readme suggests support for all versions, which appears not to be the case.

I should note that, as of a couple months ago, I could build apex just fine against pytorch 1.2.0, so it must be a recent commit that caused a breaking change. Is there any concept of apex releases, or is the master considered the only release candidate?

[kugwzk]

Is there any solution? I meet the same problem ...

[MaxMayya]

same here with pytorch 1.1.0

[FTD007]

same here

[hhaoyan]

@kugwzk @MaxMayya @FTD007 See #956