You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am curious about whether models used for merging are converted to the set dtype before merging, or if it is only the resulting merge being converted.
And if there is any loss when merging a fp16 and bf16 model?
The text was updated successfully, but these errors were encountered:
When you specify a dtype everything is converted to the specified dtype before merging, yeah.
Merging between fp16 and bf16 is potentially lossy, as both data types are capable of representing values that the other is not. This generally isn't going to be a meaningful difference though. If you're working with something super numerically sensitive, I'd recommend upcasting to fp32. I haven't found a use case that needs that kind of precision yet personally.
I am curious about whether models used for merging are converted to the set dtype before merging, or if it is only the resulting merge being converted.
And if there is any loss when merging a fp16 and bf16 model?
The text was updated successfully, but these errors were encountered: