cv2.perspectiveTransform does not work properly

System information (version)

• OpenCV => 4.3.0
• Operating System / Platform => Ubuntu 19.10

Detailed description

I have detected a possible inconsistency in cv2.perspectiveTransform.
I got an image with a polygon of known coordinates like the one below:

I know the destination coordinates of the polygon and I would like to rectify the whole image based on that transformation.
In order to do that, I first calculate the transformation between src and dst coordinates (all in the same order TopLeft, BottomLeft, BottomRight, TopRight) and then I use it to covert the corners of the image. However, this routine does not seem to work in all cases. Here is an example where the transformed coordinates seem to be completely wrong.

import cv2
import numpy as np
from PIL import Image, ImageDraw

np.set_printoptions(precision=4)
np.set_printoptions(suppress=True)

#the image dimensions
img_height=2048
img_width=2048

# image coordinates of polygon
src= np.array([[ 749.31, 1175.72],
              [ 749.31 , 364.12],
              [1171.8, 668.94],
              [1171.8 , 1105.63]])


# Create the image
image = Image.new('RGB', (img_width, img_height))
image.paste( (200,200,200), [0,0,image.size[0],image.size[1]])
draw = ImageDraw.Draw(image)
draw.line(((src[0][0],src[0][1]),(src[1][0],src[1][1]), (src[2][0],src[2][1]),(src[3][0],src[3][1]), (src[0][0],src[0][1])), width=4, fill="blue")
#image.show()


# known coordinates
dst=np.array([[0, 1000],
             [0, 0],
             [921, 0],
             [921, 1000]])

# calculate the tranformation
mat = cv2.getPerspectiveTransform(src.astype("float32"), dst.astype("float32"))

# the transformation seems to warp the polygon fine 
img_warp = Image.fromarray((
    cv2.warpPerspective(np.array(image),
                        mat,
                        (921,
                        1000))))
img_warp.show()

# new source: image corners
corners = np.array([
                [0, img_height],
                [0, 0],
                [img_width, 0],
                [img_width, img_height]
            ])

# Transform the corners of the image
corners_tranformed = cv2.perspectiveTransform(
                              np.array([corners.astype("float32")]), mat)

# These tranformed corners seems completely wrong/inverted x-axis 
print(corners_tranformed)
# [[[ -483.1003  1506.5758]
#   [ -483.1003   119.5345]
#   [-3627.2617  3817.4143]
#   [-3627.2617 -2191.3757]]]

In other cases though the transformation seems to work fine, the transformed coordinates seem fine and then I can use them to rectify the full image as shown below:

import cv2
import numpy as np
from PIL import Image, ImageDraw

np.set_printoptions(precision=4)
np.set_printoptions(suppress=True)

#the image dimensions
img_height=2048
img_width=2048


# image coordingates
src=  np.array([[ 789.72, 1187.35],
 [ 789.72, 752.75],
 [1277.35, 730.66],
 [1277.35,1200.65]])


# Create the image
image = Image.new('RGB', (img_width, img_height))
image.paste( (200,200,200), [0,0,image.size[0],image.size[1]])
draw = ImageDraw.Draw(image)
draw.line(((src[0][0],src[0][1]),(src[1][0],src[1][1]), (src[2][0],src[2][1]),(src[3][0],src[3][1]), (src[0][0],src[0][1])), width=4, fill="blue")
image.show()


# known coordinates
dst=np.array([[0, 1000],
             [0, 0],
             [1092, 0],
             [1092, 1000]])

# calculate the tranformation
mat = cv2.getPerspectiveTransform(src.astype("float32"), dst.astype("float32"))

# new source: image corners
corners = np.array([
                [0, img_height],
                [0, 0],
                [img_width, 0],
                [img_width, img_height]
            ])

# Transform the corners of the image
corners_tranformed = cv2.perspectiveTransform(
                              np.array([corners.astype("float32")]), mat)

# These tranformed corners seem ok, and in the right order.
print(corners_tranformed)
# [[[-2203.0476  3338.2544]
#   [-2203.0476 -2089.9954]
#   [ 2518.1296 -1322.9165]
#   [ 2518.1296  2571.2083]]]

x_mn = math.ceil(min(corners_tranformed[0].T[0]))
y_mn = math.ceil(min(corners_tranformed[0].T[1]))

x_mx = math.ceil(max(corners_tranformed[0].T[0]))
y_mx = math.ceil(max(corners_tranformed[0].T[1]))

print("x_mn, y_mn: ", x_mn, ", ", y_mn)
print("x_mx, y_mx: ", x_mx, ", ", y_mx)

width = x_mx - x_mn
height = y_mx - y_mn

analogy = height/1000
n_height = height/analogy
n_width = width/analogy


dst2 = corners_tranformed
dst2 -= np.array([x_mn, y_mn])
dst2 = dst2/analogy 
print("dst2 ", dst2)

mat2 = cv2.getPerspectiveTransform(corners.astype("float32"),
                                   dst2.astype("float32"))


img_warp = Image.fromarray((
    cv2.warpPerspective(np.array(image),
                        mat2,
                        (int(n_width),
                        int(n_height)))))
img_warp.show()

Is this a bug or am I missing something?
This person seems to have the same problem:
https://answers.opencv.org/question/231802/opencv-getperspectivetransform-method-gives-unclear-result/

Issue submission checklist

• I report the issue, it's not a question
• I checked the problem with documentation, FAQ, open issues,
  answers.opencv.org, Stack Overflow, etc and have not found solution
• I updated to latest OpenCV version and the issue is still there
• There is reproducer code and related data files: videos, images, onnx, etc

This is related to the vanishing point of the two lines falling inside the image. When this happens, then the corners of the image are "beyond infinity" and this is why I get the weird numbers. I added a check to see where the intersection point is and I defined my new corners accordingly. So this is not a problem of perspectiveTransform.