"LayerNormKernelImpl" not implemented for 'BFloat16'

[wangshankun]

Version:

Loading model :
RWKV-4-Pile-7B-Instruct-test1-20230124

Error:
Traceback (most recent call last):
File "chat.py", line 175, in
model = RWKV_RNN(args)
File "/opt/conda/lib/python3.8/site-packages/torch/jit/_script.py", line 272, in init_then_script
original_init(self, *args, **kwargs)
File "/root/work/ChatRWKV/src/model_run.py", line 103, in init
x = self.LN(self.w.emb.weight, self.w.blocks[0].ln0)
File "/root/work/ChatRWKV/src/model_run.py", line 111, in LN
return F.layer_norm(x, (self.args.n_embd,), weight=w.weight, bias=w.bias)
File "/opt/conda/lib/python3.8/site-packages/torch/nn/functional.py", line 2346, in layer_norm
return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)
RuntimeError: "LayerNormKernelImpl" not implemented for 'BFloat16'

[BlinkDL]

Your GPU does not support bf16. Use fp16.

[cxj01]

I had the same problem. How did you solve it?

[BlinkDL]

Ok this is indeed an issue due to a previous change. Fix it. Please try latest version.

[cxgreat2014]

cpu模式，6G模型：

RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'

[BlinkDL]

cpu模式，6G模型：

RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'

fp16 (good for GPU, does NOT support CPU) // fp32 (good for CPU) // bf16 (worse accuracy, supports CPU)