Add semantic segmentation post-processing method to MobileViT #19105
Conversation
alaradirik
changed the title
|
The documentation is not available anymore as the PR was closed or merged. |
| Args: | ||
| Converts the output of [`MobileViTForSemanticSegmentation`] into semantic segmentation maps. Only supports PyTorch.: | ||
| outputs ([`MobileViTForSemanticSegmentation`]): | ||
| Raw outputs of the model. | ||
| target_sizes (`torch.Tensor` of shape `(batch_size, 2)` or `List[Tuple]` of length `batch_size`, | ||
| *optional*): | ||
| Torch Tensor (or list) corresponding to the requested final size (h, w) of each prediction. If left to | ||
| None, predictions will not be resized. |
There was a problem hiding this comment.
Same problem as the other PR, the docstring is not properly formatted because the description of the function is after the Args and not before.
| @@ -151,3 +154,46 @@ def __call__( | |||
| encoded_inputs = BatchFeature(data=data, tensor_type=return_tensors) | |||
|
|
|||
| return encoded_inputs | |||
|
|
|||
| def post_process_semantic_segmentation(self, outputs, target_sizes: Union[TensorType, List[Tuple]] = None): | |||
There was a problem hiding this comment.
Could this be added to the Mixin instead of the FeatureExtractor?
There was a problem hiding this comment.
@hollance I agree with this but I'd prefer to do this after making sure (1) all post-processing methods of all segmentation models have consistent input arguments and naming and (2) ImageSegmentationPipeline supports all available segmentation models rather than just DETR and MaskFormer.
| Args: | ||
| outputs ([`MobileViTForSemanticSegmentation`]): | ||
| Raw outputs of the model. | ||
| target_sizes (`torch.Tensor` of shape `(batch_size, 2)` or `List[Tuple]` of length `batch_size`, | ||
| *optional*): | ||
| Torch Tensor (or list) corresponding to the requested final size (h, w) of each prediction. If left to | ||
| None, predictions will not be resized. | ||
| Returns: | ||
| semantic_segmentation: `torch.Tensor` of shape `(batch_size, 2)` or `List[torch.Tensor]` of length | ||
| `batch_size`, where each item is a semantic segmentation map of of the corresponding target_sizes entry (w, | ||
| h) if `target_sizes` is specified). Each entry of each `torch.Tensor` correspond to a semantic class id. |
There was a problem hiding this comment.
Same comments as in the other PRs.
There was a problem hiding this comment.
Thank you, this is fixed now. Post-processing uses torch for resizing and returns a list of torch tensors of shape (height, width). I also added a test for the post-processing.
| List of tuples corresponding to the requested final size (height, width) of each prediction. If left to | ||
| None, predictions will not be resized. | ||
| Returns: | ||
| semantic_segmentation: `List[torch.Tensor]` of length `batch_size`, where each item is a semantic |
There was a problem hiding this comment.
As said previously in the other PRs, this should be the return type first then the colon.
There was a problem hiding this comment.
Thanks, fixed it.
src/transformers/models/mobilevit/feature_extraction_mobilevit.py
Outdated
Show resolved
Hide resolved
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
…gface#19105) * add post-processing method for semantic segmentation Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
What does this PR do?
Adds post_process_semantic_segmentation method to
MobileViTFeatureExtractor.I will open an issue and separate PRs to make sure that
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.