Closed
Description
Hi! Thanks for the great tool!
When I run sd.cpp with some not pruned SD models it drops the error.
[ERROR] model.cpp:773 - unsupported dtype 'F64'
It seems to me a simple thing to fix, either by supporting double float internally or converting during load into single float.
I want to hear you thoughts on this situation and how to solve it.
With best regards, AG
Metadata
Metadata
Assignees
Labels
No labels
Activity
Cyberhan123 commentedon Jan 27, 2024
Can you manually set the type to f32?
FrankEscobar commentedon Feb 1, 2024
I have the same issue with a model called revAnimated_v122 but trying to convert it to F32 or F16 it gets corrupt and takes only 809Kb
Did you managed to fix it?
askmyteapot commentedon Feb 28, 2024
Having this issue too with other models in the SD1.5 family.
Could this be looked at?
grauho commentedon Mar 18, 2024
I suspect this has to do with GGML currently only having support for up to 32-bit width integer and float types in it's ggml_type. It might be possible if one is willing to accept a loss in precision to convert down to a 32-bit float, provided it does not exceed FLT_MAX, via a callback in the sdcpp load_tensors function.
grauho commentedon Mar 20, 2024
Could someone please link a model where they are having the F64 problem? I think I've put together a fix that at least seems to work with the LoRAs I have that use I64.
red-scorp commentedon Mar 20, 2024
https://civitai.com/models/8124?modelVersionId=87886
https://civitai.com/models/7371/rev-animated
Look for the bigger files, like 4-5GB. they often are FP64
grauho commentedon Mar 22, 2024
I've written a small converter program in C that re-encodes entire safetensor files that does seem to do the job, once I put in handling for those using big-endian systems I'll publish it. Having trouble getting similar logic to work a la carte at tensor loading time in sdcpp though.
grauho commentedon Mar 22, 2024
Alright here it is, feel free to give it a try. I was able to use it to convert both I64 and F64 containing models to something sdcpp could work with:
https://github.com/grauho/sdc
SA-j00u commentedon Apr 30, 2024
can you add converting to fp16 too?
grauho commentedon May 4, 2024
It now has handling to convert down into F16 as well as BF16 using the -f, --float-out switch, although with the caveat that if your system isn't using IEEE format floats and doubles it might not get the conversion right. Don't use the --replace option unless you want to risk losing data.
starcoder : update example to follow the naming convention of other e…