Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unwrap bounding box augmentation #446

Merged
merged 52 commits into from
Oct 1, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
0078f1e
Add property BoundingBox.coords
aleju Sep 27, 2019
0f744e2
Add method BoundingBox.coords_almost_equals()
aleju Sep 27, 2019
192cb3d
Add BoundingBox.almost_equals()
aleju Sep 27, 2019
ca2a0f8
Add .items property to coordinate-based OnImage classes
aleju Sep 27, 2019
98920d6
Fix duplicate test name
aleju Sep 27, 2019
daaba96
Add method Polygon.coords_almost_equals()
aleju Sep 27, 2019
0382a76
Add property Polygon.coords
aleju Sep 27, 2019
0ccdae4
Add property Keypoint.coords
aleju Sep 27, 2019
73dc126
Add Keypoint.coords_almost_equals()
aleju Sep 27, 2019
e134cf4
Fix typos
aleju Sep 27, 2019
2e6ea9e
Add Keypoint.almost_equals()
aleju Sep 27, 2019
51cf14a
Add tests for bounding box aug in Alpha
aleju Sep 27, 2019
ab11f67
Add tests for bounding box aug in AlphaElementwise
aleju Sep 28, 2019
8be7ede
Fix convert_iterable_to_string_of_types() crashing on join
aleju Sep 28, 2019
4e67889
Add testutils.assert_cbaois_equal() and .shift_cbaoi()
aleju Sep 28, 2019
2cebe84
Add tests for BBs in Flipud/Fliplr
aleju Sep 28, 2019
c3d4d83
Refactor affine matrix generation
aleju Sep 28, 2019
18c63db
Add tests for BBs, Polys, alignment in Affine scale aug
aleju Sep 28, 2019
a9e3a75
Add tests for BB/Poly aug in Affine translate
aleju Sep 28, 2019
c706637
Fix Affine coords-based aug using wrong shift
aleju Sep 28, 2019
76a8df7
Add tests for BB/Poly aug in Affine rotate & adjust shift offset
aleju Sep 28, 2019
f2ed6fb
Add tests for BB aug in PiecewiseAffine
aleju Sep 28, 2019
b338558
Fix PiecewiseAffine KP aug unaligned if KPOI empty
aleju Sep 28, 2019
c298ef5
Add tests for BB aug in PerspectiveTransform
aleju Sep 28, 2019
b6cb9fd
Add tests for BB aug in ElasticTransformation
aleju Sep 28, 2019
3940494
Add tests for BB aug in Rot90
aleju Sep 28, 2019
ddcdd91
Add tests for BB aug in Noop
aleju Sep 28, 2019
928e690
Add tests for BB aug in Sequential
aleju Sep 28, 2019
37e2aa0
Add tests for BB aug in SomeOf
aleju Sep 28, 2019
265c69d
Add tests for BB aug in Sometimes
aleju Sep 28, 2019
6d6b287
Add tests for BB aug in WithChannels
aleju Sep 28, 2019
97c1cb2
Add tests for BB aug in ChannelShuffle
aleju Sep 28, 2019
8541b77
Add missing changelog entry for PWA empty KP fix
aleju Sep 28, 2019
2651b36
Fix assert_is_iterable_of() for non-iterable inputs
aleju Sep 29, 2019
b0b8dc1
Add tests for BB and Poly aug in pooling augmenters
aleju Sep 29, 2019
9032552
Add tests for BB aug in Resize
aleju Sep 29, 2019
a4a36db
Add tests for BB aug in Pad
aleju Sep 29, 2019
80eacbf
Add tests for BB aug in Crop
aleju Sep 29, 2019
b9713fb
Add tests for BB aug in PadToFixedSize
aleju Sep 29, 2019
65675c3
Add BB aug tests to CropToFixedSize
aleju Sep 29, 2019
c6bf79d
Add tests for Poly and BB aug in KeepSizeByResize
aleju Sep 29, 2019
c8628ce
Update changelog with fix of assert_is_iterable_of()
aleju Sep 29, 2019
d33676c
Add BB aug methods to various augmenters
aleju Sep 29, 2019
7ea08bd
Add tests for BB aug in Lambda
aleju Sep 29, 2019
4142431
Add tests for BB aug in AssertLambda
aleju Sep 29, 2019
55e305d
Add tests for BB aug in AssertShape
aleju Sep 29, 2019
1dfa174
Extend changelog
aleju Sep 29, 2019
5f3fa47
Add PR ID to changelog
aleju Sep 29, 2019
ee2a389
Adapt new changelog files to fixed folder names
aleju Sep 29, 2019
e77c668
Improve flip BB aug tests BB definition
aleju Sep 29, 2019
dfd7b9b
Refactor tests
aleju Sep 29, 2019
88a287d
Add more tests for Augmenter.augment_bounding_boxes()
aleju Oct 1, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
36 changes: 36 additions & 0 deletions changelogs/master/added/20190927_unwrapped_bb_aug.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Unwrapped Bounding Box Augmentation #446

* Added property `coords` to `BoundingBox`. The property returns an `(N,2)`
numpy array containing the coordinates of the top-left and bottom-right
bounding box corners.
* Added method `BoundingBox.coords_almost_equals(other)`.
* Added method `BoundingBox.almost_equals(other)`.
* Changed method `Polygon.almost_equals(other)` to no longer verify the
datatype. It is assumed now that the input is a Polygon.
* Added property `items` to `KeypointsOnImage`, `BoundingBoxesOnImage`,
`PolygonsOnImage`, `LineStringsOnImage`. The property returns the
keypoints/BBs/polygons/LineStrings contained by that instance.
* Added method `Polygon.coords_almost_equals(other)`. Alias for
`Polygon.exterior_almost_equals(other)`.
* Added property `Polygon.coords`. Alias for `Polygon.exterior`.
* Added property `Keypoint.coords`.
* Added method `Keypoint.coords_almost_equals(other)`.
* Added method `Keypoint.almost_equals(other)`.
* Added method `imgaug.testutils.assert_cbaois_equal()`.
* Added method `imgaug.testutils.shift_cbaoi()`.
* Added internal `_augment_bounding_boxes()` methods to various augmenters.
This allows to individually control how bounding boxes are supposed to
be augmented. Previously, the bounding box augmentation was a wrapper around
keypoint augmentation that did not allow such control.
* [breaking] Added parameter `parents` to `Augmenter.augment_bounding_boxes()`.
This breaks if `hooks` was used as a *positional* argument in connection with
that method.
* [breaking] Added parameter `func_bounding_boxes` to `Lambda`. This
breaks if one relied on the order of the augmenter's parameters instead of
their names.
* [breaking] Added parameter `func_bounding_boxes` to `AssertLambda`. This
breaks if one relied on the order of the augmenter's parameters instead of
their names.
* [breaking] Added parameter `check_bounding_boxes` to `AssertShape`. This
breaks if one relied on the order of the augmenter's parameters instead of
their names.
3 changes: 3 additions & 0 deletions changelogs/master/fixed/20190928_fixed_affine_coords_aug.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
* Fixed `Affine` coordinate-based augmentation applying wrong offset
when shifting images to/from top-left corner. This would lead to an error
of around 0.5 to 1.0 pixels. #446
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
* Fixed keypoint augmentation in `PiecewiseAffine` potentially being
unaligned if a `KeypointsOnImage` instance contained no keypoints. #446
2 changes: 2 additions & 0 deletions changelogs/master/fixed/20190928_fixed_type_val.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
* Fixed `imgaug.validation.convert_iterable_to_string_of_types()` crashing due
to not converting types to strings before joining them. #446
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
* Fixed `imgaug.validation.assert_is_iterable_of()` producing a not
well-designed error if the input was not an iterable. #446
96 changes: 96 additions & 0 deletions imgaug/augmentables/bbs.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,22 @@ def __init__(self, x1, y1, x2, y2, label=None):
self.y2 = y2
self.label = label

@property
def coords(self):
"""Get the top-left and bottom-right coordinates as one array.

Returns
-------
ndarray
A ``(N, 2)`` numpy array with ``N=2`` containing the top-left
and bottom-right coordinates.

"""
arr = np.empty((2, 2), dtype=np.float32)
arr[0, :] = (self.x1, self.y1)
arr[1, :] = (self.x2, self.y2)
return arr

@property
def x1_int(self):
"""Get the x-coordinate of the top left corner as an integer.
Expand Down Expand Up @@ -762,6 +778,74 @@ def to_keypoints(self):
Keypoint(x=self.x1, y=self.y2)
]

def coords_almost_equals(self, other, max_distance=1e-4):
"""Estimate if this and another BB have almost identical coordinates.

Parameters
----------
other : imgaug.augmentables.bbs.BoundingBox or iterable
The other bounding box with which to compare this one.
If this is an ``iterable``, it is assumed to represent the top-left
and bottom-right coordinates of that bounding box, given as e.g.
an ``(2,2)`` ndarray or an ``(4,)`` ndarray or as a similar list.

max_distance : number, optional
The maximum euclidean distance between a corner on one bounding
box and the closest corner on the other bounding box. If the
distance is exceeded for any such pair, the two BBs are not
viewed as equal.

Returns
-------
bool
Whether the two bounding boxes have almost identical corner
coordinates.

"""
if ia.is_np_array(other):
# we use flat here in case other is (N,2) instead of (4,)
coords_b = other.flat
elif ia.is_iterable(other):
coords_b = list(ia.flatten(other))
else:
assert isinstance(other, BoundingBox), (
"Expected 'other' to be an iterable containing two "
"(x,y)-coordinate pairs or a BoundingBox. "
"Got type %s." % (type(other),))
coords_b = other.coords.flat

coords_a = self.coords

return np.allclose(coords_a.flat, coords_b, atol=max_distance, rtol=0)

def almost_equals(self, other, max_distance=1e-4):
"""Compare this and another BB's label and coordinates.

This is the same as
:func:`imgaug.augmentables.bbs.BoundingBox.coords_almost_equals` but
additionally compares the labels.

Parameters
----------
other : imgaug.augmentables.bbs.BoundingBox or iterable
The other object to compare against. Expected to be a
``BoundingBox``.

max_distance : number, optional
See
:func:`imgaug.augmentables.bbs.BoundingBox.coords_almost_equals`.

Returns
-------
bool
``True`` if the coordinates are almost equal and additionally
the labels are equal. Otherwise ``False``.

"""
if self.label != other.label:
return False
return self.coords_almost_equals(other, max_distance=max_distance)

def copy(self, x1=None, y1=None, x2=None, y2=None, label=None):
"""Create a shallow copy of this BoundingBox instance.

Expand Down Expand Up @@ -873,6 +957,18 @@ def __init__(self, bounding_boxes, shape):
self.bounding_boxes = bounding_boxes
self.shape = normalize_shape(shape)

@property
def items(self):
"""Get the bounding boxes in this container.

Returns
-------
list of BoundingBox
Bounding boxes within this container.

"""
return self.bounding_boxes

# TODO remove this? here it is image height, but in BoundingBox it is
# bounding box height
@property
Expand Down
90 changes: 90 additions & 0 deletions imgaug/augmentables/kps.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,21 @@ def __init__(self, x, y):
self.x = x
self.y = y

@property
def coords(self):
"""Get the xy-coordinates as an ``(N,2)`` ndarray.

Returns
-------
ndarray
An ``(N, 2)`` ``float32`` ndarray with ``N=1`` containing the
coordinates of this keypoints.

"""
arr = np.empty((1, 2), dtype=np.float32)
arr[0, :] = [self.x, self.y]
return arr

@property
def x_int(self):
"""Get the keypoint's x-coordinate, rounded to the closest integer.
Expand Down Expand Up @@ -330,6 +345,69 @@ def generate_similar_points_manhattan(self, nb_steps, step_size,
return points
return [self.deepcopy(x=point[0], y=point[1]) for point in points]

def coords_almost_equals(self, other, max_distance=1e-4):
"""Estimate if this and another KP have almost identical coordinates.

Parameters
----------
other : imgaug.augmentables.kps.Keypoint or iterable
The other keypoint with which to compare this one.
If this is an ``iterable``, it is assumed to contain the
xy-coordinates of a keypoint.

max_distance : number, optional
The maximum euclidean distance between a this keypoint and the
other one. If the distance is exceeded, the two keypoints are not
viewed as equal.

Returns
-------
bool
Whether the two keypoints have almost identical coordinates.

"""
if ia.is_np_array(other):
# we use flat here in case other is (N,2) instead of (4,)
coords_b = other.flat
elif ia.is_iterable(other):
coords_b = list(ia.flatten(other))
else:
assert isinstance(other, Keypoint), (
"Expected 'other' to be an iterable containing one "
"(x,y)-coordinate pair or a Keypoint. "
"Got type %s." % (type(other),))
coords_b = other.coords.flat

coords_a = self.coords

return np.allclose(coords_a.flat, coords_b, atol=max_distance, rtol=0)

def almost_equals(self, other, max_distance=1e-4):
"""Compare this and another KP's coordinates.

.. note ::

This method is currently identical to ``coords_almost_equals``.
It exists for consistency with ``BoundingBox`` and ``Polygons``.

Parameters
----------
other : imgaug.augmentables.kps.Keypoint or iterable
The other object to compare against. Expected to be a
``Keypoint``.

max_distance : number, optional
See
:func:`imgaug.augmentables.kps.Keypoint.coords_almost_equals`.

Returns
-------
bool
``True`` if the coordinates are almost equal. Otherwise ``False``.

"""
return self.coords_almost_equals(other, max_distance=max_distance)

def copy(self, x=None, y=None):
"""Create a shallow copy of the keypoint instance.

Expand Down Expand Up @@ -406,6 +484,18 @@ def __init__(self, keypoints, shape):
self.keypoints = keypoints
self.shape = normalize_shape(shape)

@property
def items(self):
"""Get the keypoints in this container.

Returns
-------
list of Keypoint
Keypoints within this container.

"""
return self.keypoints

@property
def height(self):
return self.shape[0]
Expand Down
21 changes: 16 additions & 5 deletions imgaug/augmentables/lines.py
Original file line number Diff line number Diff line change
Expand Up @@ -1427,8 +1427,8 @@ def almost_equals(self, other, max_distance=1e-4, points_per_edge=8):
Parameters
----------
other: imgaug.augmentables.lines.LineString
The other line string. Must be a LineString instance, not just
its coordinates.
The other object to compare against. Expected to be a
``LineString``.

max_distance : float, optional
See :func:`imgaug.augmentables.lines.LineString.coords_almost_equals`.
Expand All @@ -1439,9 +1439,8 @@ def almost_equals(self, other, max_distance=1e-4, points_per_edge=8):
Returns
-------
bool
``True`` if the coordinates are almost equal according to
:func:`imgaug.augmentables.lines.LineString.coords_almost_equals`
and additionally the labels are equal. Otherwise ``False``.
``True`` if the coordinates are almost equal and additionally
the labels are equal. Otherwise ``False``.

"""
if self.label != other.label:
Expand Down Expand Up @@ -1555,6 +1554,18 @@ def __init__(self, line_strings, shape):
self.line_strings = line_strings
self.shape = normalize_shape(shape)

@property
def items(self):
"""Get the line strings in this container.

Returns
-------
list of LineString
Line strings within this container.

"""
return self.line_strings

@property
def empty(self):
"""Estimate whether this object contains zero line strings.
Expand Down