ERROR: Failed building wheel for transformer-engine

[Weifan1226]

Hello mates!

I had a problem while installing transformer-engine wheels:

I have used two ways to install the engine: but all failed.
1:pip install git+https://github.com/NVIDIA/TransformerEngine.git@stable
2:git clone --branch stable --recursive https://github.com/NVIDIA/TransformerEngine.git
cd TransformerEngine
export NVTE_FRAMEWORK=pytorch
git submodule update --init --recursive
pip install .

Hope to get your help
Thanks!

---

“ Please use int64_t instead” Is that the problem really happen?

/root/miniconda3/lib/python3.8/site-packages/torch/include/ATen/core/boxing/impl/make_boxed_from_unboxed_functor.h:203:37: error: static assertion failed: You tried to register a kernel with an unsupported
integral input type. Please use int64_t instead.
203 | static_assert(guts::false_t::value,
| ^~~~~
error: command 'gcc' failed with exit status 1

ERROR: Failed building wheel for transformer-engine

---

Terminal output:
It seems successfully to buiild

But it faild:

[ptrendx]

Hi, which pyTorch version are you using?

[Weifan1226]

Hi, which pyTorch version are you using?

Hi, Thanks for respones.

I just find out TE is for Hopper, but I was using 4090.
I changed Megatron to V2.5 (on cuda11.1 V100) which is not using TE. And perfectly fix the problem.

Thanks

[timmoon10]

I'm glad you found a solution that worked for you. I'll just comment for future reference.

TE should work on a 4090 since Lovelace has FP8 support. This is a compilation error when TE builds some PyTorch extensions, which implies a problem with the build environment rather than with the GPU. We tend to develop using fairly recent versions of PyTorch. Looking at the line numbers in your error message, it seems you are using PyTorch 2.0.0 or 2.0.1, which are over a year old.

[Weifan1226]

I'm glad you found a solution that worked for you. I'll just comment for future reference.

TE should work on a 4090 since Lovelace has FP8 support. This is a compilation error when TE builds some PyTorch extensions, which implies a problem with the build environment rather than with the GPU. We tend to develop using fairly recent versions of PyTorch. Looking at the line numbers in your error message, it seems you are using PyTorch 2.0.0 or 2.0.1, which are over a year old.

Hi!

It is true that TE is able to work on 4090. I was using Pytorch 2.0.0 Cuda 11.8 unbuntu20.04. And today I change it into Pytorch 2.1.0, Cuda12.1,Ubuntu22.04. And TE and extensions built perferctly! I am so happy to it works well.

Thanks for your help!