Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use float32/bfloat16 for gemma3 when dtype=None #2050

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Use float32/bfloat16 for gemma3 when dtype=None #2050

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
475fc12
fix
KareemMusleh Mar 16, 2025
160fdee
Merge branch 'main' of https://github.com/unslothai/unsloth into gemm…
KareemMusleh Mar 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
fix
  • Loading branch information
@KareemMusleh
KareemMusleh committed Mar 16, 2025
commit 475fc1211ebe698df4ef2e6eb811adf528b6c514
6 changes: 5 additions & 1 deletion unsloth/models/vision.py
View file
Original file line number Diff line number Diff line change
@@ -208,7 +208,11 @@ def from_pretrained(
get_statistics() # For debugging - we use a download counter to see if environments are not breaking

if dtype is None:
dtype = torch.float16 if not SUPPORTS_BFLOAT16 else torch.bfloat16
# Inifite gradient bug for gemma3 if float16. https://unsloth.ai/blog/gemma3
if "gemma-3" in model_name.lower():
dtype = torch.bfloat16 if SUPPORTS_BFLOAT16 else torch.float32
else:
dtype = torch.bfloat16 if SUPPORTS_BFLOAT16 else torch.float16
elif dtype == torch.bfloat16 and not SUPPORTS_BFLOAT16:
logger.warning_once("Device does not support bfloat16. Will change to float16.")
dtype = torch.float16