[error] Unsloth should not depend on vLLM (Causes crash) #2591
Description
Unsloth currently imports vLLM in unsloth/models/vision.py, which causes a crash if the system doesn't have a compatible CUDA setup for vLLM. This makes Unsloth indirectly dependent on a third-party package.
Issue:
- vLLM is imported even when it's not required.
- If vLLM is installed but not compatible (e.g., CUDA mismatch), it causes a crash even if Unsloth doesn’t use any vLLM-related functionality.
- Unsloth should not be dependent on vLLM, or it should gracefully handle the absence/incompatibility of vLLM.
File:
unsloth/models/vision.py
Temporary Fix:
Commenting out the vLLM-related import lines avoids the crash (see attached screenshot).
