Applying 3D Transform to Images

[mrdotb]

Hello,

I have a collection of images that I would like to apply a 3D transformation to. I successfully achieved the desired transformation in GIMP using the 3D Transform tool, setting the x-angle to -20 degrees. Here is an example of an input image and the corresponding output:

Input:

Output:

I've also attempted to achieve the same transformation using Image and Vix.Operation libraries in a Livebook notebook, but my efforts have been unsuccessful so far. Any suggestions or guidance would be greatly appreciated.

[kipcole9]

Thanks for trying image! The operation you are describing is called warp perspective and is implemented by the function Image.warp_perspective/4. Using your image as an example:

iex> image = Image.open!("path/image.jpg")
iex> warp_from = [{404,29},{1594,46},{1872,1225},{118,1224}]
iex> warp_to =  [{342,15},{1657,26},{1802,1151},{187,1140}]
iex> infill_pixel_color = [62,86,244]
iex> {:ok, warped} = Image.warp_perspective(image, warp_from, warp_to, background: infill_pixel_color)

Which produces:

Image perspective straightening

There is also a function Image.straighten_perspective/3 that tries to straighten the image. Which can than also be cropped. For example:

iex> {:ok, bounding_box, straightened} = Image.straighten_perspective(i, [{404,29},{1594,46},{1872,1225},{118,1224}], background: [62,86,244])
{:ok, [{404, 29}, {1594, 29}, {1594, 1224}, {404, 1224}],
 %Vix.Vips.Image{ref: #Reference<0.2686404142.1520304158.259020>}}
iex> {:ok, cropped} = Image.crop(straightened, bounding_box)
{:ok, %Vix.Vips.Image{ref: #Reference<0.2686404142.1520304158.259021>}}

[mrdotb]

Thank you for your answer Image.straighten_perspective/3 give good results. However I have a lot of images to process. I am not sure if you created the warp_from by hand ?
I experiment with Evision to find the edges and warp it. It's a perfect solution.

[kipcole9]

@mrdotb would you consider posting the code you ended up with the find the edges (canny/sobel edge detection then Evision.findContours/3 perhaps?). I for one would be interested and I would like to add a generalised solution to din the "warp_from" that I can add to image.

How did you handle the blue-coloured perspective fill in the original images?

I'm learning image processing too - by implementing a library. Not the most efficient perhaps but a lot of fun!

[mrdotb]

This livebook example from evision example is very close to what I want.
It use Adaptive thresholding (I try canny as well but result was similar) and findContours
I made a livebook with the part I use and some inputs
https://gist.github.com/mrdotb/92093f3f38ee005f3af1722ec34cf4ab

It also crop and resize the image to a 1:1 ratio (which is also something I wanted) I don't need to deal with the fill.

Still the solution does not work on all the images I cheated a bit by adding a x y input to enlarge the zone. Picture of scarves are always a bit crooked.
It also fail on image with a care tag

You can check my website if you want to find more images https://tontontresors.com/hermes-scarves

Image processing is very interesting and it's so nice we can convert python code to elixir evision easily.

I will have other task related to my scarves images to do:

• auto rotate
• colors label
  ...

I will open another discussion here If it's worth

One output example from the notebook

[kipcole9]

image.autorotate/1 might address your first list item.

For the second item, "colors label", do you mean to get the segment the image by color? Get the color histogram?

Please feel free to open a discussion topic any time.

[kipcole9]

Thank you for the link to your workbook too, very interesting for me and much appreciated. My goal with Image isn't to replicate what OpenCV or libvips do. It is to make common transforms and operations more easily accessible and developer friendly for a certain class of image operations. "Image straightening" is definitely one of those. I will leverage your learning and see what I can do to make that easier for common cases in Image.