Custom image transformation?

[arbenede]

Are there any plans to add support for custom image image transformations?

[johnnychen94]

Do you mean https://juliaimages.org/ImageTransformations.jl/stable/examples/operations/swirl/#Swirl-effect-using-warp-operation?

[arbenede]

My goal is to create a custom image warp function that I can differentiate with respect to its parameters using Zygote and your other Julia package https://github.com/FluxML/DiffImages.jl. So it's not clear to me how to proceed. Should I first extend ImageTransformation.jl then add some new code to DiffImages.jl? If you have some advice I would appreciate to hear from you... By the way I opened an issue at https://github.com/FluxML/DiffImages.jl since the link to the documentation is broken.

[johnnychen94]

It should work out of the box if you're using the warp and Interpolations. The src/geometry/adjoints.jl file has added the needed rules for it. But if you're using more than that, a custom rule for some functions (which is unknown until error shows) is needed.

DiffImages was GSoC-21 project and it is still a WIP, so feel free to fork it and add whatever you need, and when you get satisfied you're welcome to submit a PR. By then we can discuss where to put the codes -- for experimental purpose, you can have everything in DiffImages.

[arbenede]

Thank you!