Fix color_splash to handle when no masks detected in model #500

robperc · 2018-04-30T04:07:46Z

For images where the model does not detect any labels it still appears to return a single mask of shape (28, 28, 1) containing all False values. The if/else statement in the color_splash function does not handle this edge case properly so when it is encountered the np.where operation fails with the error:

ValueError: operands could not be broadcast together with shapes (28, 28, 1) (<gray.shape>) (<image.shape>)

This PR fixes this by modifying the if statement to ensure that if there is a mask it has the same shape as the grayscale image. It also fixes the else branch to convert the grayscale image to uint8 as is done in the if branch. If this is not done then the following error is raised when calling imsave:

ValueError: Images of type float must be between -1 and 1.

Reported here: #500

waleedka · 2018-06-05T07:24:08Z

Good catch! However, the fix could be improved. The first dimension is the height of the image, so the image height and the mask height should always match. It turns out, however, that when there are no masks, the detection function returns a tensor of shape (28, 28, 0) rather than (image_height, image_width, 0), which is itself a bug. Your fix is relying on that behavior, so fixing that bug would break your fix.

The root cause of the issue is that the if condition is checking the first dimension, but actually the number of masks is in the last dimension. The condition should check if mask.shape[-1] > 0 but instead it's wrongly doing if mask.shape[0] > 0. The line before the condition, that squashes the masks, hides the problem because it changes the last dimension to 1 which is why you got (28, 28, 1) rather than (28, 28, 0)

I pushed a commit that fixes both bugs here a593c30

Reported here: matterport#500

* Small typo fix * loss weights * Fix multi-GPU training. A previous fix to let validation run across more than one batch caused an issue with multi-GPU training. The issue seems to be in how Keras averages loss and metric values, where it expects them to be scalars rather than arrays. This fix causes scalar outputs from a model to remain scalar in multi-GPU training. * Replace keep_dims with keepdims in TF calls. TF replaced keep_dims with keepdims a while ago and now shows a warning when using the old name. * Headline typo fix in README.md Fixed the typo in the headline of the README.md file. "Spash" should be "Splash" * Splash sample: fix filename and link to blog post * Update utils.py * Minor cleanup in compute_overlaps_masks() * Fix: color_splash when no masks are detected Reported here: matterport#500 * fix typo fix typo * fix "No such file or directory" if not use: "keras.callbacks.TensorBoard" * Allow dashes in model name. Print a message when re-starting from saved epoch * Fix problem with argmax on (0,0) arrays. Fix matterport#170 * Allow configuration of FPN layers size and top-down pyramid size * Allow custom backbone implementation through Config.BACKBONE This allows one to set a callable in Config.BACKBONE to use a custom backbone model. * modified comment for image augmentation line import to include correct 'pip3 install imgaug' instructions * Raise clear error if last training weights are not foundIf using the --weights=last (or --model=last) to resume trainingbut the weights are not found now it raises a clear error message.

Reported here: matterport#500

* Small typo fix * loss weights * Fix multi-GPU training. A previous fix to let validation run across more than one batch caused an issue with multi-GPU training. The issue seems to be in how Keras averages loss and metric values, where it expects them to be scalars rather than arrays. This fix causes scalar outputs from a model to remain scalar in multi-GPU training. * Replace keep_dims with keepdims in TF calls. TF replaced keep_dims with keepdims a while ago and now shows a warning when using the old name. * Headline typo fix in README.md Fixed the typo in the headline of the README.md file. "Spash" should be "Splash" * Splash sample: fix filename and link to blog post * Update utils.py * Minor cleanup in compute_overlaps_masks() * Fix: color_splash when no masks are detected Reported here: matterport#500 * fix typo fix typo * fix "No such file or directory" if not use: "keras.callbacks.TensorBoard" * Allow dashes in model name. Print a message when re-starting from saved epoch * Fix problem with argmax on (0,0) arrays. Fix matterport#170 * Allow configuration of FPN layers size and top-down pyramid size * Allow custom backbone implementation through Config.BACKBONE This allows one to set a callable in Config.BACKBONE to use a custom backbone model. * modified comment for image augmentation line import to include correct 'pip3 install imgaug' instructions * Raise clear error if last training weights are not foundIf using the --weights=last (or --model=last) to resume trainingbut the weights are not found now it raises a clear error message. * Fix Keras engine topology to saving * Fix load_weights() for Keras versions before 2.2 Improve previous commit to not break on older versions of Keras. * Update README.md * Add custom callbacks to model training Add an optional parameter for calling a list of keras.callbacks to be add to the original list. * Add no augmentation sources Add the possibility to exclude some sources from augmentation by passing a list of sources. This is useful when you want to retrain a model having few images. * Improve previous commit to avoid mutable default arguments * Updated Coco Example * edit loss desc * spellcheck config.py * doublecheck on config.py * spellcheck utils.py * spellcheck visualize.py * Links to two more projects in README * Add Bibtex to README * make pre_nms_limit configurable * Make pre_nms_limit configurable * Made compatible to new version of VIA JSON format VIA has changed JSON formatting in later versions. Now instead of a dictionary, "regions" has a list, see the issue matterport#928 * Comments to explain VIA 2.0 JSON change * Fix the comment on output shape in RPN * Bugfix for MaskRCNN creating empty log_dir that breaks find_last() - Current implementation creates self.log_dir in set_log_dir() function, which creates an empty log directory if none exists. This causes find_last() to fail after creating a model because it finds this new empty directory instead of the previous training directory. - New implementation moves log_dir creation to the train() function to ensure it is only created when it will be used. * Added automated epoch recognition for Windows. (matterport#798) Unified regex expression for both, Linux and Windows. * Fixed tabbing issue in previous commit * bug fix: the output_shape of roi_gt_class_ids is incorrect * Bug fix: inspect_balloon_model.ipynb Fix bugs of not showing boxes in 1.b RPN Predictions. TF 1.9 introduces "ROI/rpn_non_max_suppression/NonMaxSuppressionV3:0", so the original code can't work. * Apply previous commit to the other notebooks * Fixed comment on GPU_COUNT (matterport#878) Fixed comment on GPU_COUNT * add IMAGE_CHANNEL_COUNT class variable to config to make it easier to use Mask_RCNN for non 3-channel images * Additional comments for the previous commit * Link to new projects in README * Tiny correction in README. * Adjust PyramidROIAlign layer shape comment For PyramidROIAlign's output shape, use pool_height and pool_width instead of height and width to avoid confusion with those of feature_maps. * fix output shape of fpn_classifier_graph 1. fix the comment on output shape in fpn_classifier_graph 2. unify NUM_CLASSES and num_classes to NUM_CLASSES 3. unify boxes, num_boxes, num_rois, roi_count to num_rois 4. use more specific POOL_SIZE and MASK_ POOL_SIZE to replace pool_height and pool_width * Fix PyramidROIAlign output shape As discussed in: matterport#919 * Fix comments in Detection Layer 1. fix description on window 2. fix output shape of detection layer * use smooth_l1_loss() to reduce code duplication * A wrapper for skimage resize() to avoid warnings skimage generates different warnings depending on the version. This wrapper function calls skimage.tranform.resize() with the right parameter for each version. * Remove unused method: append_data()

Reported here: matterport#500

Reported here: matterport/Mask_RCNN#500

robperc added 2 commits April 29, 2018 20:39

Fix color_splash to handle when no masks detected in model

d08b1fb

Remove redundant condition in if statement

595faaa

waleedka added a commit that referenced this pull request Jun 5, 2018

Fix: color_splash when no masks are detected

a593c30

Reported here: #500

waleedka closed this Jun 5, 2018

LexLuc pushed a commit to LexLuc/Mask_RCNN that referenced this pull request Jun 5, 2018

Fix: color_splash when no masks are detected

c55a761

Reported here: matterport#500

LackesLab pushed a commit to LackesLab/Mask_RCNN that referenced this pull request Aug 24, 2018

Fix: color_splash when no masks are detected

3120ee0

Reported here: matterport#500

Cpruce pushed a commit to Cpruce/Mask_RCNN that referenced this pull request Jan 17, 2019

Fix: color_splash when no masks are detected

73df77f

Reported here: matterport#500

withyou53 pushed a commit to withyou53/mask_r-cnn_for_object_detection that referenced this pull request Sep 27, 2023

Fix: color_splash when no masks are detected

50fcd66

Reported here: matterport/Mask_RCNN#500

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix color_splash to handle when no masks detected in model #500

Fix color_splash to handle when no masks detected in model #500

robperc commented Apr 30, 2018

waleedka commented Jun 5, 2018

Fix color_splash to handle when no masks detected in model #500

Fix color_splash to handle when no masks detected in model #500

Conversation

robperc commented Apr 30, 2018

waleedka commented Jun 5, 2018