Why self-conditioning?

[Silverster98]

Are there some works or explanations demonstrating the effectiveness of self-conditioning implementation? Or it is just an empirical trick?

denoising-diffusion-pytorch/denoising_diffusion_pytorch/denoising_diffusion_pytorch.py

Lines 656 to 668 in 6e8a0f2

	# if doing self-conditioning, 50% of the time, predict x_start from current set of times
	# and condition with unet with that
	# this technique will slow down training by 25%, but seems to lower FID significantly
	x_self_cond = None
	if self.self_condition and random() < 0.5:
	with torch.no_grad():
	x_self_cond = self.model_predictions(x, t).pred_x_start
	x_self_cond.detach_()
	# predict and take gradient step
	model_out = self.model(x, t, x_self_cond)

[yiyixuxu]

see paper here https://arxiv.org/abs/2208.04202

[tuttyfrutyee]

This (self-conditioning) does not work for me btw (train loss insists on not decreasing), but the data I work with might be too noisy compared to natural images.