Foreground extraction has a wide number of applications in computer vision. As technology advances, the resolution of your everyday photo also increases. This is problematic for older traditional foreground extraction methods as the computation time increases tremendously when working with high resolution images.
This repo proposes a foreground extraction algorithm that reduces the time needed to process high resolution images while still achieving a decent result. Inspiration of algorithm came from here.
The algorithm takes in three images as input, the source image, the foreground seed image and the background seed image. The seed images give the algorithm a very basic guideline when deciding whether a pixel is a part of the foreground or background.
To handle the resolution of the image in a reasonable time, we construct an image pyramid by downsampling the source image to create a low resolution version of the image. We use the traditional grabcut algorithm to get a rough foreground mask of the source image and upsample the mask back to the source image's original resolution. This mask is labelled the initial mask. More information regarding grabcut here.
Using the initial mask, we can generate improved foreground and background seeds using the contour of the mask. We proceed to examine patches along the contour to get a more refined cut. Each patch will contain pixels that are already marked by the improved foreground and background seeds and the cut performed will be based off of the color and location of each pixel within the patch. The algorithm assumes that if a pixel has a similar color intensity as the cluster of seeded foreground pixels within the patch, or if it is located close to the cluster, then it should considered as a foreground pixel. The same logic applies for background pixels. We speed up processing by spawning four threads process multiple patches concurrently.
Once each patch is refined, its intensity values are copied back onto an eroded version of the initial mask to create the refined mask. The source image is then cut out using the refined mask.
python run.py -s [path to source image] -f [path to foreground seed image] -b [path to background seed image] -t [threshold value for smallest contour size] -o [path to output image]
python run.py -s test_images/test2/source.jpg -f test_images/test2/foreground.png -b test_images/test2/background.png -t 5000 -o test_images/test2/out.png
