Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

imresize/imrotate fixed point #129

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

imresize/imrotate fixed point #129

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
515f11b
add imresize with fixed_point
ginkulv Jun 6, 2021
342c9fd
add struct FixedPoint
ginkulv Jun 7, 2021
8c60c63
add examples of imresize with fixed point
ginkulv Jun 7, 2021
1d3fee9
rename FixedPoint to CenterPoint
ginkulv Jun 8, 2021
81ffd49
refactor imresize with FixedPoint
ginkulv Jun 30, 2021
cb837d7
add some tests and fix the description for CenterPoint
ginkulv Jul 2, 2021
dd9d5c7
add some tests
ginkulv Jul 23, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion src/ImageTransformations.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,10 @@ export
warpedview,
InvWarpedView,
invwarpedview,
imrotate
imrotate,
CenterPoint

include("centerpoint.jl")
include("autorange.jl")
include("interpolations.jl")
include("warp.jl")
Expand Down
16 changes: 16 additions & 0 deletions src/centerpoint.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
"""
CenterPoint(dims) -> cp

Create a fixed point which can be used in `imresize` and `imrotate`
functions in order to keep the value in this point the same, i.e.,

```jldoctest
img[cp] == imgr[cp]
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this jldoctest is definitely broken. Using

Suggested change
```jldoctest
img[cp] == imgr[cp]
```
```julia
img[cp] == imgr[cp]
```

would be fine.

The more concerning issue here is that maybe there isn't such a fixed point concept at all. All we're introducing is a warp center point. Looks like it is this property that holds true:

CartesianIndices(img)[cp] == CartesianIndices(imgr)[cp]

I'm in a bad state here and don't have enough bandwidth to think this carefully. Can you double-check it?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I could make such a bad mistake, sorry. I will test it carefully.

Copy link
Author

@ginkulv ginkulv Jun 8, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't notice 'at all' and got the meaning of your comment wrong.

All we're introducing is a warp center point.

Honestly, that's what I was thinking about. Not to disturb your state, but can you give any tips of how this idea looks in your head? Othewise I'm afraid I don't understand.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the back-and-forth and the inaccuracy.

In https://evizero.github.io/Augmentor.jl/dev/operations/#Affine-Transformations, the scale/resize is no different from imresize. I was thinking of a generic solution to the Zoom operator there where the center point is invariant. The term "fixed point" showed itself in my head when I wrote down #121.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delayed response, got stuck with exams. Unfortunately, I still can't fully understand your concerns. I made a poor visualization of what it does right now when I create a new image with imresize(img, FixedPoint(100, 100); ratio=0.5).
resizing
To me it looks like it works as expected and implements the Zoom operator, except it doesn't crop the image. Hopefully, my point is clear and makes sense.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! I'll recheck next weekend (also busy with school stuff). In the meantime, if you can add up some tests when you're available it would be great.

"""
struct CenterPoint{N}
p::CartesianIndex{N}
end

CenterPoint(dims::Dims{N}) where N = CenterPoint{N}(CartesianIndex(dims))
CenterPoint(dims::Int64...) = CenterPoint(Tuple(dims))
23 changes: 22 additions & 1 deletion src/resizing.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ upsample/downsample the image `img` to a given size `sz` or axes `inds` using in
The output size is `ceil(Int, size(img).*ratio)`. If `ratio` is larger than `1`, it is
an upsample operation. Otherwise it is a downsample operation. `ratio` can also be a tuple,
in which case `ratio[i]` specifies the resize ratio at dimension `i`.
- `method::InterpolationType`:
- `method::InterpolationType`:
specify the interpolation method used for reconstruction. conveniently, `methold` can
also be a `Degree` type, in which case a `BSpline` object will be created.
For example, `method = Linear()` is equivalent to `method = BSpline(Linear())`.
Expand All @@ -83,6 +83,10 @@ imresize(img, (1:256, )) # 256*768
imresize(img, ratio = 0.5) #256*384
imresize(img, ratio = (2, 1)) # 1024*768

# pass `FixedPoint`
imresize(img, (256, 384), FixedPoint(10, 15)) # 256*384, img[10,15] == imgr[10,15]
imresize(img, FixedPoint(10, 15), ratio = 0.5) # 256*384, img[10,15] == imgr[10,15]

# use different interpolation method
imresize(img, (256, 384), method=Linear()) # 256*384 bilinear interpolation
imresize(img, (256, 384), method=Lanczos4OpenCV()) # 256*384 OpenCV-compatible Lanczos 4 interpolation
Expand Down Expand Up @@ -120,6 +124,23 @@ function imresize(original::AbstractArray{T,N}, new_inds::Indices{N}; kwargs...)
end
end

function imresize(original::AbstractArray{T,N}, new_size::Dims{N}, center_point::CenterPoint{N}; kwargs...) where {T,N}
Tnew = imresize_type(first(original))
cp = center_point.p
checkbounds(Bool, original, cp) || error("given point $cp is out of range")
topleft = firstindex.(Ref(original), Tuple(1:N))
offset = cp.I .- new_size .* (cp.I .- topleft) .÷ size(original) .- 1
resized = OffsetArray(similar(original, Tnew, new_size), offset)
imresize!(resized, original; kwargs...)
resized[cp] = original[cp]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that resized[cp] can be different from original[cp] because of the warp method we're using. But I don't think we need to overwrite resized[cp] here.

Looks like there's no fixed point at all 😢

resized
end

function imresize(original::AbstractArray{T,N}, center_point::CenterPoint{N}; ratio, kwargs...) where {T,N}
all(ratio .> 0) || throw(ArgumentError("ratio $ratio should be positive"))
new_size = ceil.(Int, size(original) .* ratio) # use ceil to avoid 0
imresize(original, new_size, center_point; kwargs...)
end
# To choose the output type, rather than forcing everything to
# Float64 by multiplying by 1.0, we exploit the fact that the scale
# changes correspond to integer ratios. We mimic ratio arithmetic
Expand Down