Compare image for duplicate

[ridzria]

Let’s say I have an image already uploaded and stored in the system.

When a user upload the same image for instance, is there a way to compare with existing list of images by the same user and see if it is duplicate indeed. If not then allow the user to upload otherwise throw an error.

Is there comparison of images for duplicate feature is available? If not this feature is awesome to have.

[tielur]

@ridzria you might be better off doing something like a md5 checksum comparison or something along those lines

[kipcole9]

@ridzria, the function Image.dhash/1 may be what you're after. It generates a variation of a perceptual hash and does a reasonable job of checking if images are "the same" irrespective of size or format. You'll see the references to the original implementations in the docs.

Please note that the original generates 64-bit hashes. My function generates 512-bit hashes. Which means I've done something different in my implementation but I haven't taken the time to dive deeper into it.

Please do let me know if Image.dhash/1 does what you need?

Usage

def kinda_same?(image_1, image_2) do
  Image.dhash(image_1) == Image.dhash(image_2)
end

[ridzria]

That means let's say I have an image url which I wanted to compare.

There are 2 scenarios.

1. 2 image urls
2. 1 image url and 1 file that I'm uploading

Scenario 1:

image_url_1 = "https://server/image1.jpg"
image_url_2= "https://server/image2.jpg"

def compare_image(image_url_1, image_url_2)
{
      image_1 = get_image(image_url_1)
      image_2 = get_image(image_url_2)

      case Image.dhash(image_1) == Image.dhash(image_2) do
      true ->
      {:error, :duplicate_image}

      false ->
      {:ok, :safer_to_upload}
      end
}

def get_image(url)
{
     %HTTPoison.Response{body: image_data} = HTTPoison.get!(url)
     {:ok, image_data} = Image.from_binary(image_data)
     image_data
}

Scenario 2:

image_url_1 = "https://server/image1.jpg"

def compare_image_with_upload_image(image_url_1, upload_image_path)
{
      image_1 = get_image(image_url_1)
      image_2 = upload_image_path |> Base.encode64()

      case Image.dhash(image_1) == Image.dhash(image_2) do
      true ->
      {:error, :duplicate_image}

      false ->
      {:ok, :safer_to_upload}
      end
}

Scenario 1: I have 2 urls of same image with different resolutions. When I compare I get false. If the image of same resolution I get true. This works only for same resolutions? Or we can tweak to compare any same image with various resolutions?

[kipcole9]

In your scenario 2 image_2 = upload_image_path |> Base.encode64() wouldn't return a valid image so I'm surprised the code in that scenario works? Perhaps that code is just an extract?

For scenario 1, perceptual hashes aren't perfect - but they don't really care about resolution. The first thing the hash does is to resize the image to an 8x8 size. You might find Image.hamming_distance/2 to be more useful if your images are very similar, but not the same (same meaning same hash).

Something like:

# See Image.hamming_distance/2
@similarity_threshold 10

def compare_image(image_url_1, image_url_2) do
  image_1 = get_image(image_url_1)
  image_2 = get_image(image_url_2)

  if Image.hamming_distance(image_1, image_2) < @similarity_threshold do
   {:ok, :safer_to_upload}
  else
   {:error, :duplicate_image}
  end
end

[ridzria]

This isn't working for me. I get duplicate_image when I tried for 2 urls.

def get_image(image_url) do
    response = HTTPoison.get!(image_url)

    case response.status_code do
      200 ->
        image_data = response.body

        case image_data do
          image_data ->
            try do
              {:ok, image_data} = Image.from_binary(image_data)
              image_data
            rescue
              _ ->
                {:error, :timeout}
            end

          {:error, :timeout} ->
            nil
        end

      _ ->
        nil
    end
end

iex> cat = get_image(cat_image_url)
%Vix.Vips.Image{ref: #Reference<0.2089409803.3355050010.40428>}
iex> tiger = get_image(tiger_image_url)
%Vix.Vips.Image{ref: #Reference<0.2089409803.3355050010.40258>}
iex> if Image.hamming_distance(cat, tiger) < 10 do {:ok, :safer} else {:error, :duplicate} end
{:error, :duplicate}

[kipcole9]

There is no perfect solution here. But I am willing to keep working on it. Can you share the URLs of the two images? Or attach the images here?

[ridzria]

Sure. Here is the link for cat and tiger.

https://images.hdqwalls.com/wallpapers/cute-kitten-4k-im.jpg

https://news.stanford.edu/wp-content/uploads/2021/02/GettyImages-627540386.jpg

[kipcole9]

Please note that I have just pushed a commit that returns the Image.dhash/1 as a 64-bit hash (as originally intended) rather than a 512-bit one.