The TF installation fails with `No such file #include <cudnn_frontend.h>`

[kaixih]

With the latest code, it seems we can no longer build the code in the TF containers. After executing pip install ., I got:

FAILED: common/CMakeFiles/transformer_engine.dir/fused_attn/fused_attn_fp8.cu.o
      /usr/local/cuda/bin/nvcc -forward-unknown-to-host-compiler -Dtransformer_engine_EXPORTS -I/home/workspace/repo_zoo/TransformerEngine/transformer_engine -I/home/workspace/repo_zoo/TransformerEngine/transformer_engine/common/include
-I/usr/local/cuda/targets/x86_64-linux/include -I/home/workspace/repo_zoo/TransformerEngine/transformer_engine/../3rdparty/cudnn-frontend/include -I/tmp/tmpy_w9ikze/common/string_headers -isystem=/usr/local/cuda/include --threads 4 --exp
t-relaxed-constexpr -O3 -O3 -DNDEBUG --generate-code=arch=compute_70,code=[compute_70,sm_70] --generate-code=arch=compute_80,code=[compute_80,sm_80] --generate-code=arch=compute_89,code=[compute_89,sm_89] --generate-code=arch=compute_90,
code=[compute_90,sm_90] -Xcompiler=-fPIC -std=c++17 -MD -MT common/CMakeFiles/transformer_engine.dir/fused_attn/fused_attn_fp8.cu.o -MF common/CMakeFiles/transformer_engine.dir/fused_attn/fused_attn_fp8.cu.o.d -x cu -c /home/workspace/re
po_zoo/TransformerEngine/transformer_engine/common/fused_attn/fused_attn_fp8.cu -o common/CMakeFiles/transformer_engine.dir/fused_attn/fused_attn_fp8.cu.o
      In file included from /home/workspace/repo_zoo/TransformerEngine/transformer_engine/common/fused_attn/fused_attn_fp8.cu:10:
      /home/workspace/repo_zoo/TransformerEngine/transformer_engine/common/fused_attn/utils.h:14:10: fatal error: cudnn_frontend.h: No such file or directory
         14 | #include <cudnn_frontend.h>
            |          ^~~~~~~~~~~~~~~~~~

It seems we need the similar setup_tensorflow_extension function as for the pytorch or paddle in the setup.py, where the /opt/tensorflow/cudnn-frontend/include/ needs to be added.

@trevor-m Can you take a look when you have bandwidth?

Also curious: is such installation break captured by our CI/CD?

[ksivaman]

cudnn_frontend is a submodule in TE and needs to be checked out when building from source. You can run git submodule update --init --recursive before building.

[nouiz]

The error doesn't help end user to understand what is going on or how to fix it.
Could we detect in the setup.py that the submodule is missing and raise an understannable error?
We have been many to hit this issue.

[kaixih]

Thanks for the pointer. I used this installation guide link from the homepage: https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/installation.html, which seems to be less informative. Shall we at least update it?

[timmoon10]

In some sense this is a problem with Git submodules and Git users should be in the practice of checking out submodules. I say that, but I've run into this error multiple times myself. Blame Linus. I'm thinking we should modify the CMake so that it checks for cudnn_frontend.h and errors out with a reminder to checkout submodules.

[nouiz]

Exactly. And with the command line that should fix it.
Most git user don't know well git submodule, so we shouldn't expect them how to fix it even if we tell them what the issue is.

[kaixih]

errors out with a reminder to checkout submodules

Yes, this is exactly what we need. Some simple hint of submodule checking should be sufficient. When I first came across the above error, I didn't realize the header has already been included in the repo, as a submodule tho.

[timmoon10]

#307 prints a warning message when the cuDNN front-end is not detected during building.