Fix attribute error when loading FusedAdamBuilder()

[rraminen]

This PR is to avoid the below error when loading FusedAdamBuilder() extension.

    deepspeed.ops.op_builder.FusedAdamBuilder().load()
  File "/root/DeepSpeed/op_builder/builder.py", line 453, in load
    return self.jit_load(verbose)
  File "/root/DeepSpeed/op_builder/builder.py", line 492, in jit_load
    if not self.build_for_cpu and self.enable_bf16:
AttributeError: 'FusedAdamBuilder' object has no attribute 'enable_bf16'

cc: @jithunnair-amd

[jithunnair-amd]

@rraminen thanks for the fix. It looks like this issue should occur for both CUDA and ROCm?

[rraminen]

@rraminen thanks for the fix. It looks like this issue should occur for both CUDA and ROCm?

I have not tested it on CUDA, but it looks like it would occur on CUDA too.

[jithunnair-amd]

@jeffra This is being done in the constructor for OpBuilder class, even though the enable_bf16 attributed is really only checked for CUDAOpBuilder:

DeepSpeed/op_builder/builder.py

Line 492 in 82991df

if not self.build_for_cpu and self.enable_bf16:

The issue is that the initialization of that attribute happens inside https://github.com/microsoft/DeepSpeed/blob/82991df05419828d65cbb658efc01b50c135ea00/op_builder/builder.py#LL517C9-L517C32, which is only called for CUDA.

Would you prefer to initialize this in OpBuilder or CUDAOpBuilder?

[jithunnair-amd]

cc @tjruwase