Inserting image with arbitrary tranformation

[Rodrigodd]

I have a project where I need to convert a PDF file to another format, and back to PDF.

When I extract the images in the PDF (with get_image_info), I receive the transformation matrix of the Image:

{'bbox': (134.80172729492188,
          275.05450439453125,
          460.4739685058594,
          566.8353271484375),
 'bpc': 8,
 'colorspace': 3,
 'cs-name': 'DeviceRGB',
 'digest': b'\xe8\x05\x89\xcb)f\xa3N e`\x18\xda\xed\xf0\xe7',
 'height': 412,
 'number': 0,
 'size': 29575,
 'transform': (-283.699462890625,
               -134.12982177734375,
               -41.9727783203125,
               157.6510009765625,
               460.4739685058594,
               409.184326171875),
 'width': 988,
 'xref': 7,
 'xres': 96,
 'yres': 96}

But I don't know how I can add the image back to the PDF when rebuilding it. The documentation says that there are two methods for adding images:

• Page.insert_image, but at max it can only do rotations multiples of 90; and
• Page.show_pdf_page, that can do any arbitrary rotation, but not flipping or skewing. It also does not explain exactly how I must use it.

I am using this PDF as a test file: rotated_image.pdf

[JorjMcKie]

That's a hard one.
You are right: both insertion methods do not offer arbitrary treatment of the image before insertion.

• We have currently no intention to support arbitrary rotation angles with page.insert_image() - we will stay with int multiples of 90°.
• Method page.show_pdf_page() does support arbitrary rotation angles. So you can use it for your purpose - after having your image converted to a 1-page PDF (which is an easy thing to do).
• If you need special treatments like flipping or tilting, then you must indeed use some other package before insertion, like Pillow.

To insert an image insertion via show_pdf_page() you must convert the image to a 1-page PDF first. For example:

img = fitz.open("image.file")  # open image file as a fitz.Document (!) using its filename
# the above works in the same way if your image is a memory area (bytes object)

imgpdf = fitz.open("pdf", img.convert_to_pdf())  # convert to a 1-page PDF

# insert image via show_pdf_page on a target page:
tar_page.show_pdf_page(rect, imgpdf, 0, rotate=angle)

[Rodrigodd]

If you need special treatments like flipping or tilting, then you must indeed use some other package before insertion, like Pillow.

Yes, but I prefer avoiding that if possible, because it would be a lossy transformation, and I know that the PDF itself can encode these transformations.

We have currently no intention to support arbitrary rotation angles with page.insert_image().

Is there any strong reasoning for that?

Ideally I would like to have something like the morph parameter in the Shape API, or even a Shape.insert_image, as I am already using Shape to draw all the other elements in the page.

But in any case, I think flipping/skewing the image with Pillow and then rotating it with show_pdf_page may suffice for now.

[JorjMcKie]

Is there any strong reasoning for that?

We have lots of higher priority stuff to do. Strong enough?

[Rodrigodd]

Fair enough. Was only curious if it was a design or technical limitation of how it is currently implemented.

[JorjMcKie]

Ok, understand. This position was deemed to be ok vis-a-vis the fact that we do have show_pdf_page.

[AlexandruIca]

This is understandable, and in this case they might be equivalent. But I just want to note that converting an image to a PDF first, and then using show_pdf_page is not quite identical to using insert_image. The latter can receive a transparency mask for the image, meanwhile converting to PDF just ignores that completely. If one needs both arbitrary rotations and transparent images, is there any option?

[Rodrigodd]

Unfortunately, the previous solution was not enough for me. The show_pdf_page was able to apply an arbitrary rotation followed by an arbitrary scaling, but I was at least hoping that it could apply the scaling before the rotation.

But I realize that I could chain two calls to show_pdf_page in order to achieve what I want. In fact, I could achieve any orientation-preserving transformation.

With a good dose of linear algebra (and try and error), I came up with the following function:

def add_tranformed_image(page, image_filename, transform):
    from math import tan, atan, sqrt, sin, cos

    # Because the Rect given to show_pdf_page cannot have negative width or
    # height, PyMuPDF cannot flip a image. We flip it using PIL instead,
    # although it reencodes the image.
    det = transform[0] * transform[3] - transform[1] * transform[2]
    if det == 0:
        raise ValueError("Singular matrix")
    if det < 0:
        # flip image using PIL
        img = Image.open(image_filename)
        img = ImageOps.flip(img)

        stream = io.BytesIO()
        img.save(stream, format="PNG")
        
        # flip the transform
        transform = [transform[0], transform[1],
                    -transform[2], -transform[3],
                    transform[4] + transform[2], transform[5] + transform[3]]
        det = -det
        
        # open the image with fitz
        img = fitz.open(stream=stream, filetype="png")
    else:
        img = fitz.open(image_filename)

    width = img[0].bound().width
    height = img[0].bound().height

    # the given transform assumes the image is 1x1.
    transform = [
        transform[0] / width, transform[1] / width,
        transform[2] / height, transform[3] / height,
        transform[4], transform[5]
    ]
    det = transform[0] * transform[3] - transform[1] * transform[2]

    a, b, c, d, dx, dy = transform

    # the solution below is not valid for the entire domain, so apply a initial
    # random rotation as a workaround
    initial_rotation = 0
    while abs(b*c) < det * 1e-6 or a*d*(a*d - b*c) < 0 or ((-2*a*d + b*c - 2*sqrt(a*d*(a*d - b*c)))/(b*c)) < 0:
        initial_rotation = (initial_rotation + random.rand() * math.pi * 2) % (math.pi * 2)
        co = cos(initial_rotation)
        si = sin(initial_rotation)
        transform = [
             a*co + c*si,  b*co + d*si,
            -a*si + c*co, -b*si + d*co,
            dx, dy
        ]
        a, b, c, d, dx, dy = transform

    # Solution of the equation T = S*R*K, for a given T, where
    # T = [a, b, c, d] # final transformation matrix
    # S = [sx, 0, 0, sy] # scaling matrix
    # R = [cos(theta), sin(theta), -sin(theta), cos(theta)] # rotation matrix
    # K = [kx, 0, 0, ky] # scaling matrix
    #
    # Generated with SymPy
    sy = 1
    sx = a*sy*tan(2*atan(sqrt((-2*a*d + b*c - 2*sqrt(a*d*(a*d - b*c)))/(b*c))))/c
    kx = (-a*d + b*c - sqrt(a*d*(a*d - b*c)))/(b*sy*sqrt(-2*a*d/(b*c) + 1 - 2*sqrt(a*d*(a*d - b*c))/(b*c)))
    ky = (a*d + sqrt(a*d*(a*d - b*c)))*(a*d - b*c + sqrt(a*d*(a*d - b*c)))/(a*b*c*sy*(-2*a*d/(b*c) + 1 - 2*sqrt(a*d*(a*d - b*c))/(b*c)))
    theta = 2*atan(sqrt(-(2*a*d - b*c + 2*sqrt(a*d*(a*d - b*c)))/(b*c)))

    # Normalize the solution to have positive sx, sy, kx, ky
    if sx < 0:
        sx = -sx
        kx = -kx
        theta = -theta
    if sy < 0:
        sy = -sy
        ky = -ky
        theta = -theta
    if kx < 0 and ky < 0:
        kx = -kx
        ky = -ky
        theta = theta + math.pi

    assert(kx > 0 and ky > 0)

    def rotate(x,y,theta):
        return x * cos(theta) + y * sin(theta), x * -sin(theta) + y * cos(theta)
    
    def rotate_bbox(bbox, theta):
        points = [
            rotate(bbox[0], bbox[1], theta),
            rotate(bbox[2], bbox[1], theta),
            rotate(bbox[0], bbox[3], theta),
            rotate(bbox[2], bbox[3], theta),
        ]
        return [
            min(points, key=lambda x: x[0])[0],
            min(points, key=lambda x: x[1])[1],
            max(points, key=lambda x: x[0])[0],
            max(points, key=lambda x: x[1])[1],
        ]

    # Draw the image to new document, with the initial_rotation and them scaled by S

    imgpdf = fitz.open("pdf", img.convert_to_pdf())

    # We need to scale the bounding box of the image, after the image is rotated
    rot_bbox = rotate_bbox([0, 0, width, height], initial_rotation)
    rot_bbox = [
        rot_bbox[0] * sx, rot_bbox[1] * sy,
        rot_bbox[2] * sx, rot_bbox[3] * sy
    ]
    rot_width = rot_bbox[2] - rot_bbox[0]
    rot_height = rot_bbox[3] - rot_bbox[1]

    img2pdf = fitz.open()
    img2page = img2pdf.new_page(width=rot_width, height=rot_height)
    img2page.show_pdf_page(fitz.Rect(0, 0, rot_width, rot_height), imgpdf, 0, rotate=initial_rotation*180/math.pi, keep_proportion=False)

    # Draw the image with rotated by R, and the scaled by K

    rot_center = rotate(-rot_bbox[0], -rot_bbox[1], theta)
    rot_bbox = rotate_bbox([0, 0, rot_width, rot_height], theta)
    rot_bbox = [
        rot_bbox[0] - rot_center[0], rot_bbox[1] - rot_center[1],
        rot_bbox[2] - rot_center[0], rot_bbox[3] - rot_center[1]
    ]

    rot_bbox = [
        rot_bbox[0]*kx + dx, rot_bbox[1]*ky + dy,
        rot_bbox[2]*kx + dx, rot_bbox[3]*ky + dy
    ]

    page.show_pdf_page(fitz.Rect(rot_bbox), img2pdf, 0, rotate=theta * 180 / math.pi, keep_proportion=False)

Of course, this function could be cleaned a lot more, but I am not willing of investing too much into it, this solution is too workaroundish.

For my previous example, we could use it like this:

pdf = fitz.open()
page = pdf.new_page(width=596, height=842)

transform = [247.46170043945312,
               -61.38348388671875,
               110.96814727783203,
               136.8866729736328,
               29.47948455810547,
               85.38616943359375]

add_tranformed_image(page, "filename.png", transform)

pdf.save("result.pdf")