Skip to content

v-code01/remapalign

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

remapalign

kornia.geometry.remap resamples an image at a caller-supplied pixel-space map. It normalizes that map to grid_sample coordinates with the 2p/(size-1) - 1 convention, the one grid_sample reads correctly only with align_corners=True, but remap's align_corners defaults to None and it resolves that to False. The normalization and the resampling therefore use different pixel-center conventions, so an identity map, which must return the image unchanged, comes back scaled by size/(size-1) and shifted half a pixel. The default remap(image, map_x, map_y) silently corrupts its output.

The code

kornia, 0.8.3 and current main (identical):

# kornia/geometry/conversions.py, normalize_pixel_coordinates: the (size-1) convention
factor = 2.0 / (hw - 1)
return factor * pixel_coordinates - 1

# kornia/geometry/transform/imgwarp.py, remap (line 492 / 581 on main)
def remap(image, map_x, map_y, ..., align_corners: Optional[bool] = None, normalized_coordinates=False):
    ...
    map_xy_norm = normalize_pixel_coordinates(map_xy, height, width)   # (size-1) convention
    if align_corners is None:
        align_corners = False                                          # line 559-560 / 648-649
    ...
    return F.grid_sample(image, map_xy_norm, align_corners=align_corners, ...)

normalize_pixel_coordinates maps pixel p to 2p/(size-1) - 1. grid_sample with align_corners=False maps a normalized n back to pixel ((n+1)/2)*size - 0.5. Composing the two sends p to p*size/(size-1) - 0.5, not p. With align_corners=True the unnormalization is ((n+1)/2)*(size-1), which inverts the normalization exactly. The normalization is hardcoded to the size-1 convention, so the align_corners=False default can never be self-consistent for pixel inputs.

What it does

Driving the real remap on a 5x5 image with an identity pixel map (fp64):

identity pixel map through real kornia remap:
  default (align_corners=None->False) : max|out-img| = 18.000
  align_corners=True                  : max|out-img| = 0.000
  default output row0 [0.0, 0.375, 1.0, 1.625, 1.0]  vs input row0 [0.0, 1.0, 2.0, 3.0, 4.0]

The identity map is create_meshgrid(H, W, normalized_coordinates=False), exactly the pixel-space map the signature documents. The default remap returns a geometrically distorted image; the same call with align_corners=True returns the input to machine precision. The base case isolates the fault: align_corners=True matches the (size-1) normalization, so it is exact, which proves the distortion comes from the default flag and not from the map or the interpolation. Every internal caller works around the default by forcing align_corners=True (geometry/calibration/undistort.py, the fisheye augmentation), and the docstring example passes align_corners=True, so the mismatch is already known to the surrounding code; only the public default is wrong.

Scope

The trigger is automatic on the documented default path: remap(image, map_x, map_y) with pixel coordinates, the mainstream call, silently returns a scaled and half-pixel-shifted image with no warning, matching neither the function's own dst(x, y) = src(map_x(x, y), map_y(x, y)) contract nor OpenCV's cv2.remap semantics. The fix is to default align_corners to True, matching the normalization convention, as every internal caller already does.

Layout

  • excerpt.py: the (size-1) normalization, the align_corners=None -> False resolution, and the grid_sample call quoted with the flags that name the fault (Apache-2.0).
  • remap.py: the normalize/unnormalize round-trip on a 1D coordinate, identity only when both use the same convention, and p*size/(size-1) - 0.5 for the default.
  • consequence.py: the real remap on an identity map, default versus align_corners=True, and the distorted first row.
  • test_remapalign.py: the model round-trip is the identity only for align_corners=True and is a scale-plus-shift otherwise; the real default remap corrupts the identity map while align_corners=True is exact.

Reproduce

python remap.py
python consequence.py
python test_remapalign.py

The normalization and the align_corners default are quoted from current main; the tensors are produced by the real remap. The fix is to default align_corners to True so the resampling uses the same pixel-center convention as the normalization and an identity map reproduces the image.

About

kornia remap normalizes the pixel map with the (size-1)/align_corners=True convention but calls grid_sample with align_corners=False by default, so an identity map silently returns a scaled, half-pixel-shifted image

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages