Keypoints relocate wrongly when rotated

[offchan42]

I observed that when I use iaa.Affine(rotate="anything that is not zero"), the keypoints shifted wrongly on the small image.

In the following image, the 3rd plot is rotated by 180 degrees, but somehow the keypoints are not in the center of the circle. The problem is more noticeable when you resize the image further to 30x30.

Here's the minimal code to reproduce the issue:

import cv2 as cv
import numpy as np
import imgaug as ia
from imgaug import augmenters as iaa
from imgaug.augmentables.kps import KeypointsOnImage

def enlarge_and_plot(img, koi):
    plot = cv.resize(img, (600, 600))
    plot = koi.on(plot).draw_on_image(plot, size=10)
    return plot

# creating big image and big keypoints
img = np.ones((500,500,3), dtype=np.uint8) * 255
coords = [100,100,200,200,300,300,400,400,100,400]  # list of XY coordinates
coords = np.float32(coords).reshape((-1, 2))
for coord in coords:
    cv.circle(img, tuple(coord), 10, (0,0,0), thickness=3)
koi = KeypointsOnImage.from_xy_array(coords, img.shape)
plot1 = enlarge_and_plot(img, koi)

# make small image and small keypoints
img = cv.resize(img, (45, 45))
koi = koi.on(img)
plot2 = enlarge_and_plot(img, koi)

# rotate small image and small keypoints
print(koi)
img, koi = iaa.Affine(rotate=180)(image=img, keypoints=koi)
print(koi)
plot3 = enlarge_and_plot(img, koi)

ia.imshow(np.hstack([plot1, plot2, plot3]))

I don't know if this is a bug or it's simply my mistake of how I'm using the library. Please give me insight. @aleju

But I think it's kind of an off-by-one error. I see that the first coordinate (100,100) gets mapped to (9,9) on 45x45 image, which is correct, but after rotation, the coordinate gets mapped to (35,35) which should be (36,36) if I understand correctly.
Here's the output of the program:

KeypointsOnImage([Keypoint(x=9.00000000, y=9.00000000), Keypoint(x=18.00000000, y=18.00000000), Keypoint(x=27.00000191, y=27.00000191), Keypoint(x=36.00000000, y=36.00000000), Keypoint(x=9.00000000, y=36.00000000)], shape=(45, 45, 3))
KeypointsOnImage([Keypoint(x=35.00000000, y=35.00000000), Keypoint(x=26.00000000, y=26.00000000), Keypoint(x=16.99999809, y=16.99999809), Keypoint(x=8.00000000, y=8.00000000), Keypoint(x=35.00000000, y=8.00000000)], shape=(45, 45, 3))

If this is the case, it means that you won't notice this bug in a big image because one pixel wrong is not visible to the naked eyes

[offchan42]

My current hack is to do something like this:

img = "some small image"
img, koi = iaa.Affine(rotate=180)(image=img, keypoints=koi.on((10000,10000)))
koi = koi.on(img)

This will make the off-by-one error insignificant.
But this doesn't look elegant, especially when I have a list of KeypointsOnImage and I have to convert each of them to big keypoints.

[aleju]

Can you try the latest version from master and see if it works then as expected?
Luckily, it looks like patch #446 in commit c706637 has already fixed that one.

This is the output that I got with master:

KeypointsOnImage([Keypoint(x=9.00000000, y=9.00000000), Keypoint(x=18.00000000, y=18.00000000), Keypoint(x=27.00000191, y=27.00000191), Keypoint(x=36.00000000, y=36.00000000), Keypoint(x=9.00000000, y=36.00000000)], shape=(45, 45, 3))
KeypointsOnImage([Keypoint(x=36.00000000, y=36.00000000), Keypoint(x=27.00000000, y=27.00000000), Keypoint(x=17.99999809, y=17.99999809), Keypoint(x=9.00000000, y=9.00000000), Keypoint(x=36.00000000, y=9.00000000)], shape=(45, 45, 3))

Thanks for the detailed bug report though!

[offchan42]

OK Nice.!
How do I properly install from master when I have already installed it via conda install imgaug -c conda-forge before?
I tried conda uninstall imgaug and it seems like it's trying to remove too many things e.g. numpy py-opencv, etc.
Actually, I've uninstalled using that command and then I can't do pip install git+https://github.com/aleju/imgaug.git. It says missing numpy.get_include() which seems to be a missing library issue. I think it's because the uninstall command removes too many things.

[offchan42]

OK. I figured it out now. I just have to use pip uninstall imgaug instead of conda uninstall imgaug before installing from master. This conda uninstall command is quite dangerous as it breaks all my numpy and pandas stuff. I have to reinstall things again.

It seems the new code is working good.

So here's the last question:
When will the bug fix gets released to conda or pip so that I don't have to install from git repo?

By the way, your library is awesome it helps my project a lot. So thank you!

[aleju]

I hope that the next release will happen in two weeks or so. Not entirely sure yet.

[offchan42]

OK. I think my issue is solved. I can just use the latest version from master.