Detect speech bubbles in Swift

The idea with this project is to try to detect speech bubbles on comics/mangas using Swift. Especially using Vision, Core Image and Core Graphics.

Steps

1. Read the image
2. Clean the image
3. Detect contours
4. Filter contours to discard unreasonable contours
5. Use the contours bounds to clip the image and the contour poligon path to mask the invers of the speech bubble.

After that, the image should be ready for OCR

Progress

Read the Image.

The image I will be using is a page of the manga ブラックジャックによろしく

Clean the Image

For this step I'm applying multiple CIFilters. The speech bubbles should be really obvious after this step.

Detect Contours

A VNDetectContoursRequest seems to be doing a great job of detecting contours

Filter Contours

Filter contours that have an area of at least 0.004, and no bigger than 0.4 and that have a proportion of area/perimeter of at least 0.02

Clean the Images

With the contours identified, it's possible to crop and mask individual bubbles. To help on cleaning the bubble border, the path is stroked with white color.

TODO:

• It's possible that some contours inside contours aren't being checked.

Resources

• https://heartbeat.fritz.ai/new-in-ios-14-vision-contour-detection-68fd5849816e
• https://developer.apple.com/videos/play/wwdc2020/10673
• https://www.thepythoncode.com/article/contour-detection-opencv-python
• https://github.com/damishshah/comic-book-reader