what is the difference between z_q_x and z_q_x_bar?

[PanXiebit]

hi @rakhimovv, thanks for your perfect work on video generation.

lvt/vidgen/modeling/vq/vq_embedding.py

Lines 37 to 38 in 0899c21

	z_q_x_, indices = vq_st(z_e_x_, self.embedding.weight.detach())
	z_q_x = z_q_x_.permute(0, 3, 1, 2).contiguous()

lvt/vidgen/modeling/vq/vq_embedding.py

Lines 61 to 64 in 0899c21

	z_q_x_bar_flatten = torch.index_select(self.embedding.weight,
	dim=0, index=indices)
	z_q_x_bar_ = z_q_x_bar_flatten.view_as(z_e_x_)
	z_q_x_bar = z_q_x_bar_.permute(0, 3, 1, 2).contiguous()

In the codes of the VQEmbedding, I am confused on this line, it seems that z_q_x and z_q_z_bar are the same for z_q(x) which is the produced codes. Hope for your response.

[rakhimovv]

Hi, @PanXiebit ! z_q_x and z_q_z_bar will differ if you use EMA for updating the dictionary. As you can see z_q_x_bar is obtained using self.embedding.weight, but the last one is updated inside the if clause

[PanXiebit]

@rakhimovv thanks!