Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass,RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

[lionday]

"I don't want to use 8-bit training. I hope to use fp16 training. After commenting out these two lines, there was an error. How should I modify it? In addition, how much VRAM is needed for fp16 and int8 training respectively?"

model = AutoModelForCausalLM.from_pretrained(
args.model_path,
use_auth_token=True,
use_cache=not args.no_gradient_checkpointing,
# torch_dtype=torch.float16,
# load_in_8bit=True,
device_map={"": Accelerator().process_index},
)
#model = prepare_model_for_int8_training(model)

[Shrishml]

I got the same error while trying to comment 8 bit quantization

[CEfanmin]

Me too.