[bug]: diffusers pipeline should run with autocast disabled

[keturn]

We currently have some code in generator.base that turns on autocast when float16 is in use:

InvokeAI/ldm/invoke/devices.py

Lines 24 to 29 in 3a17246

	def choose_autocast(precision):
	'''Returns an autocast context or nullcontext for the given precision string'''
	# float16 currently requires autocast to avoid errors like:
	# 'expected scalar type Half but found Float'
	if precision == 'autocast' or precision == 'float16':
	return autocast

We do not want to be using autocast while executing the diffusers pipeline.

Ideally we can take that out entirely, but if it's required for some of the legacy ckpt_generator code, we may be able to work around it by either moving the autocast further down to that code, or protecting the diffusers code by wrapping it in with autocast(enabled=False).

Code that fails with type errors when autocast is disabled (e.g. Input type (float) and bias type (c10::Half) should be the same) needs to be cleaned up so the tensors are their correct types. Preferably as early to their creation as possible, instead of converting them on every access.

Additional context

This is related to and possibly blocking other issues:

• [bug]: SD 2.x models return only black in float16 #2329
• [bug]: diffusers - FP16 doesn't work for MPS devices, causes low level driver/library crash #2336

[lstein]

I removed the autocast scoping and proceeded to fix all the Input type (float) and bias type (c10::Half) should be the same errors and am quite sure that the model is running with torch.dtype==torch.fp16. Unfortunately I'm still getting black images when running at half-precision on the SD-2.1 768 model.