train flux_controlnet code need to update

[Johnson-yue]

This code is just for instantx controlnet, not suitable xlabs

diffusers/examples/controlnet/train_controlnet_flux.py

Lines 1258 to 1266 in aa14f09

	control_latents = vae.encode(control_values).latent_dist.sample()
	control_latents = (control_latents - vae.config.shift_factor) * vae.config.scaling_factor
	control_image = FluxControlNetPipeline._pack_latents(
	control_latents,
	control_values.shape[0],
	control_latents.shape[1],
	control_latents.shape[2],
	control_latents.shape[3],
	)

reference FluxControlNetPipeline code:

diffusers/src/diffusers/pipelines/flux/pipeline_flux_controlnet.py

Lines 907 to 922 in aa14f09

	# xlab controlnet has a input_hint_block and instantx controlnet does not
	controlnet_blocks_repeat = False if self.controlnet.input_hint_block is None else True
	if self.controlnet.input_hint_block is None:
	# vae encode
	control_image = retrieve_latents(self.vae.encode(control_image), generator=generator)
	control_image = (control_image - self.vae.config.shift_factor) * self.vae.config.scaling_factor
	# pack
	height_control_image, width_control_image = control_image.shape[2:]
	control_image = self._pack_latents(
	control_image,
	batch_size * num_images_per_prompt,
	num_channels_latents,
	height_control_image,
	width_control_image,
	)

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.