Improve semantic segmentation models #14355
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sgugger
approved these changes
Nov 11, 2021
| @@ -76,6 +77,26 @@ def prepare_feat_extract_dict(self): | |||
| } | |||
|
|
|||
|
|
|||
| def prepare_semantic_single_inputs(): | |||
| ds = load_dataset("hf-internal-testing/fixtures_ade20k", split="test") | |||
There was a problem hiding this comment.
Let's give a better name than just ds, we try to avoid such short names in the codebase.
LysandreJik
approved these changes
Nov 17, 2021
NielsRogge
force-pushed
the
fix_segformer
branch
from
36cd4fa
to
ed14f3e
Compare
Albertobegue
pushed a commit
to Albertobegue/transformers
that referenced
this pull request
Jan 27, 2022
* Improve tests * Improve documentation * Add ignore_index attribute * Add semantic_ignore_index to BEiT model * Add segmentation maps argument to BEiTFeatureExtractor * Simplify SegformerFeatureExtractor and corresponding tests * Improve tests * Apply suggestions from code review * Minor docs improvements * Streamline segmentation map tests of SegFormer and BEiT * Improve reduce_labels docs and test * Fix code quality * Fix code quality again
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
SegformerFeatureExtractortrying to access non-existent.ndimattribute #14332) for SegFormersemantic_ignore_index, which defaults to 255. The loss functions of semantic segmentation models typically use 255 instead of -100. The reason for this is that some datasets include a 0 in the annotated segmentation maps to indicate "background", however it can be that background is not included in any of the labels of the dataset. e.g. ADE20k has 150 labels, but "background" is not included. Therefore, one reduces the labels of all segmentation maps by 1 value, and replaces the 0 by 255 as shown here. It's only after that that images get resized using PIL. However, if we replace values by -100, PIL can't read these images, and will thrown an error.segmentation_mapstoBeitFeatureExtractor, and add corresponding tests.To do:
SegformerFeatureExtractorcurrently includes thealign,do_random_cropanddo_padarguments at initialization, however I wonder whether it's maybe better to remove those, and only include the bare minimum in the feature extractors (similar toViTFeatureExtractor) to get started (i.e. resizing, center cropping, normalizing). Things like random cropping and padding is maybe already a bit too much and makes the feature extractors more complex. It's also not easy to determine good default values for this feature extractor; should it randomly crop + pad by default, or not?SegformerFeatureExtractor, if one agrees on this.preprocessor_config.jsonof the semantic segmentation models on the hub.