Clip Skip Setting when training

[jsbach-jung]

Hi, I've searched extensively about this, and plan to test it when the holiday craze is over.

Are the following statements true?

• If clip skip is set to 1 in your Automatic UI Settings, at the time you train your model, it will target Clip Skip 1, much like SD 1.5
• If clip skip is set to 2 in your Automatic UI Settings, it will target the training on Clip Skip 2, like for NAI

I noticed this recently when I fine tuned a model (a mixed model from Civitai, similar to HassanBlend) of a person and noticed it produced "correct" results only on Clip Skip 2, and I realized I had left it on 2 in UI Settings while I had been experimenting with a NAI-based model right before training.

I was surprised because I'd mixed onto that mixed model before and it was trained on to Clip 1, but now I was only seeing results on Clip 2.

If this is not correct, is there a way we can "target" a particular Clip when we train, is it available in the parameters? Thanks.

[leppie]

Are the following statements true?

I believe both are true when training the text encoder.

Relevant code is here:

sd_dreambooth_extension/dreambooth/finetune_utils.py

Lines 211 to 217 in 633ca33

	clip_skip = shared.opts.CLIP_stop_at_last_layers
	if clip_skip <= 1:
	encoder_hidden_states = text_encoder(input_ids)[0]
	else:
	enc_out = text_encoder(input_ids, output_hidden_states=True, return_dict=True)
	encoder_hidden_states = enc_out['hidden_states'][-clip_skip]
	encoder_hidden_states = text_encoder.text_model.final_layer_norm(encoder_hidden_states)