Skip to content

Conversation

@CuriousDolphin
Copy link
Member

@CuriousDolphin CuriousDolphin commented Feb 21, 2025

Optimize Segmentation Mask Storage and Add Bounding Box Information to inference result

CONTEXT

The segmentation models were storing masks at full image resolution, leading to unnecessary memory usage and larger payload sizes in API responses or base64 encoding. Additionally, semantic and instance segmentation outputs lacked bounding box information (easily derivable from binary masks), limiting their usefulness in downstream applications.

KEY CHANGES

  • Crop segmentation masks to object bounding boxes before base64 encoding
  • Add bounding box coordinates to semantic and instance segmentation outputs
  • Optimize mask processing with OpenCV for better performance
  • Rename vision utility functions for better clarity
  • Add mask_to_xyxy utility function for bbox extraction from masks
  • Add Integer bbox coordinates insteaf of float

IMPACT

  • Reduces API response payload size for segmentation results and latency
  • Affects all segmentation model outputs (both semantic and instance)
  • Improves memory efficiency in mask storage and processing
  • Provides more complete object information in segmentation results

@github-actions
Copy link

github-actions bot commented Feb 21, 2025

Coverage

Coverage Report •
FileStmtsMissCoverMissing
__init__.py100100% 
local_model.py630100% 
ports.py249896%454–456, 458, 460–462, 464
remote_model.py1403674%227–229, 234, 238–239, 241–242, 246–247, 275–283, 285–288, 292–295, 299–304, 306–307, 364
runtime.py1739743%32, 40–41, 145, 149, 153, 163, 165–166, 168–170, 172, 175–177, 180, 183–185, 188, 191–192, 195–196, 199, 224–228, 230–231, 234–237, 239–240, 242, 246–249, 253–254, 256–258, 260–264, 266–267, 269–271, 275, 285–286, 296–300, 302, 304–305, 307–313, 317–320, 324–325, 327–328, 330–334, 336–337, 339–340, 342, 352–353
utils
   vision.py96495%54, 126, 164, 194
TOTAL106915285% 

Tests Skipped Failures Errors Time
116 1 💤 0 ❌ 0 🔥 6.644s ⏱️

@CuriousDolphin CuriousDolphin marked this pull request as ready for review February 21, 2025 16:28
Copy link
Contributor

@giuseppeambrosio97 giuseppeambrosio97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@giuseppeambrosio97 giuseppeambrosio97 merged commit 5b6497d into main Feb 24, 2025
9 checks passed
@giuseppeambrosio97 giuseppeambrosio97 deleted the feat/crop-masks-with-bbox branch February 24, 2025 11:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[postprocess] crop masks and add bbox (masks to bbox) to non detection models [inference] make bbox integer instead of float

3 participants