Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Black regions of the clustered images causes Google Reverse Image Search to Fail #12

Open
DragonflyRobotics opened this issue Jun 22, 2022 · 0 comments
Assignees
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@DragonflyRobotics
Copy link
Member

Description
The clustering algorithm leaves the parts of the masked images that aren't part of the clustered class black. This makes it incredibly hard for Google to classify them. Using some clever algorithm to remove these black spots can help a lot.

To Reproduce
Steps to reproduce the behavior:

  1. Put this image in your script directory:
    image

  2. Run this:

from MAGIST.Vision.UnsupervisedModels.img_cluster import RoughCluster

cluster = RoughCluster("config.json")

imgs = cluster.unsupervised_clusters(6, "Input.jpg", (150, 150), "Clusters")


from MAGIST.Utils.WebScraper.google import GoogleScraper

scraper = GoogleScraper("config.json")

labels = []

for i in imgs:
	label = scraper.reverse_image_search(i)
	labels.append(label)

print(labels)

Expected behavior
Ideally, it should mask out nearby, similar pixels but it doesn't. This leaves massive, unstructured black gaps that cause a lot of issues when reverse searching.

Screenshots
image

image

Additional context
Google generally returns language or night when there is too much black. The solution would be something like this:

  1. Find each masked image's edge pixels.
  2. Compute all possible lines that can be formed from edge pixels.
  3. Crop image at that line if all pixels in the line are black.

This is super computationally intensive, however.

@DragonflyRobotics DragonflyRobotics added bug Something isn't working help wanted Extra attention is needed labels Jun 22, 2022
@DragonflyRobotics DragonflyRobotics added this to To do in MAGIST v0.1.0 AlphaBird via automation Jun 22, 2022
@DragonflyRobotics DragonflyRobotics self-assigned this Jun 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
No open projects
Development

No branches or pull requests

1 participant