-
Notifications
You must be signed in to change notification settings - Fork 560
Open
Description
The script scripts/run_multimodal.py currently has two distinct issues that affect its reliability and usability in documented environments.
1. Critical Bug: Script Hangs in Non-Interactive Environments
- Problem: The script calls
image.show()after loading each image, which attempts to open a GUI window. When the script is run inside a Docker container or any other headless environment, this call blocks execution indefinitely, causing the script to hang. - Impact: This bug makes the script unusable in one of its primary intended environments.
2. Code Quality: Brittle Checkpoint Loading
- Problem: The script uses a manual
model.load_state_dict(torch.load(...))call to load weights. This method is not as robust as the built-in helper function. - Impact: It fails to leverage the
model.load_weights()method, which is designed to handle both single-file and sharded checkpoints automatically. This makes the script less flexible and not aligned with the library's best practices.
Proposed Solution
To resolve these issues, the following changes are proposed:
- Remove the
image.show()call to ensure the script runs correctly in all environments. - Refactor the weight loading logic to use
model.load_weights()for improved robustness and maintainability.
Metadata
Metadata
Assignees
Labels
No labels