Batched Transforms And OutOfMask Sampling #563
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The TPS cdist function creates enormous symmetric matrices for large images (since there are a lot of indices to sample inside large images). Therefore, this commit allows an optional kwarg batch_size for warping, whereby the points to warp get passed in smaller batches. By default, this is not used.
If the target has points that are nearly coincident, the coefficients matrix is rank deficient, and therefore not invertible. Therefore, we only take the inverse on the full-rank matrix and drop any singular values that are less than a very small value (close to zero).
|
Did it solve the weird warpings you were getting? |
|
No :( But I just copied it from Yuxiang, so I guess it fixed it for On 3/10/15, Joan Alabort-i-Medina notifications@github.com wrote:
|
|
OK, good. I think in general this means that the TPS warp is not exact any more, i.e. landmark correspondences are not strictly kept... You could also try using a slightly bigger |
This allows you to batch the input to a transform, which is more general than the batched input for image warping that I had before. This is particularly useful for transforms that have large intermediate data structures such as PWA and TPS.
Now, sampling is still batched, but there is an explicit sampling method. Also, batching is now removed as it is done by the transforms themselves.
Just abuse kwargs. order shouldn't be specified to BooleanImage, but since we reuse the Image superclass implementation it is, therefore, just ignore the order parameter by abusing kwargs.
Default cval was 0.0, might as well be False, as this is more accurate for the boolean image data type.
Given the new sampling behaviour, there is now a new definition for the behaviour of warping masked images. It will **only** warp if the target mask is True everywhere that the reference mask wants to sample. Otherwise, it will throw an OutOfMaskSampleError, which will fix problems for things like building AAMs with masked Images. If you want to sample the whole image, you as_unmasked. If you want to warp the mask, use warp_to_shape on the mask, warp the unmasked image, then create a masked image and set the warped mask on the warped image.
Also, only set landmarks if it had landmarks
Only consider those points that are inside the bounding box of the landmarks. This is a massive performance improvement for large images like breaking bad with a small masked region.
Also, fix but about catching and throwing the TriangleContainmentError in PWA. Simply needed to override the _apply_batched method for PWA.
Missed the case whereby all the points are INSIDE the bounding box, aka the landmarks ARE a box. This should set all points to true. Also, the bounds are inclusive on the top end, so we add +1 to the max bounds to be correct. Added tests for these cases and checked on master.
This allows 'baking' of the mask. Useful for visualization.
|
See description, I have edited it to include all the new changes I've made. I've tried to add tests for any of the new functionality I added. |
| Alignment.__init__(self, source, target) | ||
| if self.n_dims != 2: | ||
| raise ValueError('TPS can only be used on 2D data.') | ||
| if kernel is None: | ||
| kernel = R2LogR2RBF(source.points) | ||
| self._min_singular_val = min_singular_val |
There was a problem hiding this comment.
@patricksnape I think this would be better non '_' - imagine users would want to be able to check this parameter after constructing warps in the future
Loses underscore so that is a tunable parameter after the transform has been built.
| sampled_pixels : (`n_points`, `n_channels`) `ndarray` | ||
| The interpolated values taken across every channel of the image. | ||
| """ | ||
| from menpo.shape import PointCloud |
There was a problem hiding this comment.
@patricksnape do we need this import inline? maybe if we do add a comment to clarify there is a circular import issue
|
+1 |
patricksnape
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Original
Allow for large warping and account for an instability that can occur in TPS warps. See commit messages for more details.
Edit
This PR has become a bit more heavy handed, basically adding a bunch of stuff that I need.
samplemethod to images for sampling from arbitrary floating point locations.constrain_mask_to_landmarksto only consider the bounding box of the landmarks, which thus ends up transforming many less points. Constraining the BreakingBad image goes from 7s to 700ms on my machine with this change.warp_to_mask,warp_to_shapeandconstrain_mask_to_landmarksas_unmaskedwhich allows setting the value of the unmasked pixels. This is useful for 'baking' the mask for visualization, without needing to use Matplotlib.