Q1) SophiaH, AdaHessian optimizers give RuntimeError: ~ tensors does not require grad and does not have a grad_fn in compute_hutchinson_hessian().
create_graph must be set True when calling backward(). here's an example.