fused_layer_norm_cuda

[song201216]

Hi, I am very interested in your research. When I reproduced your code, the following problems appeared, and I felt very confused.
Traceback (most recent call last):
File "generate_samples.py", line 326, in
main()
File "generate_samples.py", line 321, in main
model = setup_model(args)
File "generate_samples.py", line 52, in setup_model
model = get_model(args)
File "E:\CogView-main\pretrain_gpt2.py", line 65, in get_model
model = GPT2Model(num_layers=args.num_layers,
File "E:\CogView-main\model\gpt2_modeling.py", line 91, in init
self.transformer = mpu.GPT2ParallelTransformer(num_layers,
File "E:\CogView-main\mpu\sparse_transformer.py", line 460, in init
[get_layer(layer_id) for layer_id in range(num_layers)])
File "E:\CogView-main\mpu\sparse_transformer.py", line 460, in
[get_layer(layer_id) for layer_id in range(num_layers)])
File "E:\CogView-main\mpu\sparse_transformer.py", line 441, in get_layer
return GPT2ParallelTransformerLayer(
File "E:\CogView-main\mpu\sparse_transformer.py", line 283, in init
self.input_layernorm = LayerNorm(hidden_size, eps=layernorm_epsilon)
File "E:\CogView-main\mpu\sparse_transformer.py", line 42, in init
super().init(*args, **kwargs)
File "D:\software\Anaconda3\envs\syq\lib\site-packages\apex\normalization\fuse
d_layer_norm.py", line 133, in init
fused_layer_norm_cuda = importlib.import_module("fused_layer_norm_cuda")
File "D:\software\Anaconda3\envs\syq\lib\importlib_init_.py", line 127, in
import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'fused_layer_norm_cuda'

[poorfriend]

重新安装apex
https://blog.csdn.net/li_jiaoyang/article/details/117431876