Fix default parameter of text_encoder_projection_dim in _get_add_time_ids

[viiika]

fix default parameter of text_encoder_projection_dim from None to 1280

[b-burton]

I think this change also needs to be made to made to pipline_controlnet_inpaint_sd_xl.py (and possibly others): https://github.com/huggingface/diffusers/blob/6db33337a49c2f20db1b8d2ad069cca10c552c68/src/diffusers/pipelines/controlnet/pipeline_controlnet_inpaint_sd_xl.py#L1072C5-L1117C1

Same issue that text_encoder_projection_dim=None should be text_encoder_projection_dim=1280

see:

diffusers/src/diffusers/pipelines/controlnet/pipeline_controlnet_inpaint_sd_xl.py

Lines 1072 to 1081 in 6db3333

	def _get_add_time_ids(
	self,
	original_size,
	crops_coords_top_left,
	target_size,
	aesthetic_score,
	negative_aesthetic_score,
	dtype,
	text_encoder_projection_dim=None,
	):

---

However I think there is an additional issue here,

diffusers/src/diffusers/pipelines/controlnet/pipeline_controlnet_inpaint_sd_xl.py

Lines 1089 to 1091 in 6db3333

	passed_add_embed_dim = (
	self.unet.config.addition_time_embed_dim * len(add_time_ids) + text_encoder_projection_dim
	)

Should probably be changed to something like:

if text_encoder_projection_dim is not None:
     passed_add_embed_dim = (
           self.unet.config.addition_time_embed_dim * len(add_time_ids) + text_encoder_projection_dim
      )
else:
     passed_add_embed_dim = self.unet.config.addition_time_embed_dim * len(add_time_ids)

In order to handle len(add_time_ids) always being an int and then trying to add a NoneType (TypeError: unsupported operand type(s) for +: 'int' and 'NoneType')

[hlky]

When calling _get_add_time_ids we always pass in text_encoder_projection_dim, as you've pointed out the argument is actually required so I'm not sure why it was made optional, maybe this should be changed instead cc @yiyixuxu

[yiyixuxu]

yeah agree with @hlky

ok to remove = None part

[viiika]

yeah agree with @hlky

ok to remove = None part

but for sdxl, it seems all usage of text_encoder_projection_dim are set to 1280. So let users set a value might be redundant and confusing as many of the dependent codes are written with its default form.

[hlky]

We'll need to run make fix-copies after this change

[viiika]

this may damage some external dependencies

[hlky]

We consider functions prefixed with _ to be non-public, however any external dependency using _get_add_time_ids must be passing text_encoder_projection_dim as the function doesn't work without, removing = None will have no effect, the function is called in the same way.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.