-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix TorchVision channel preprocessing #3173
Conversation
@jimthompson5802 FYI |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very nice work on this!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice fast fix! Just one (optional) nit.
ludwig/features/image_feature.py
Outdated
self, | ||
metadata: TrainingSetMetadataDict, | ||
torchvision_transform: Optional[torch.nn.Module] = None, | ||
transform_metadata: Optional[Dict[str, Any]] = None, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Consider breaking up transform_metadata
into separate fields: transform_height
, transform_width,
transform_num_channels`, or used a dataclass for stronger field guarantees and typing:
@dataclass
class ImageTransformMetadata:
height: int
width: int
num_channels: int
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice suggestion– I wasn't sure that torchscript supported this, but apparently it does! pytorch/pytorch#72901
This PR introduces automatic channel resizing for torchvision models to ensure that images fed into torchvision encoders always have 3 channels. Closes #3170.