Memory leaks (?) when inserting images

[daniel-kukiela]

Please provide all mandatory information!

Describe the bug (mandatory)

When inserting images to the pdf files opened in a loop, memory usage constantly increases.

To Reproduce (mandatory)

Example code:

import fitz
import cv2
import numpy
import io
import numpy as np

# Infinitive loop
while True:
    # Empty document for this example
    with fitz.open() as doc:
        # Let's also instert 100 empty pages
        for i in range(100):
            doc.insertPage(i)
            page = doc[i]

            # Create image rectangle
            image_rectangle = fitz.Rect(0, 0, 10, 10)

            # Example image - 1000x1000 px zaved as png into byte stream
            image = np.zeros((100, 100, 3))
            _, image_numpy_bytes = cv2.imencode(".png", image, [cv2.IMWRITE_PNG_COMPRESSION, 9])
            image_bytes = io.BytesIO(image_numpy_bytes)

            # Inserting image - test with this commented out to see no memory leaks, then uncomment and check again
            #page.insertImage(image_rectangle, stream=image_bytes)

Expected behavior (optional)

Release all resources on doc.close()

Your configuration (mandatory)

• Ubuntu 20.04
• Python 3.8, 64bit
• PyMuPDF 1.18.2 installed from pip

Additional context (optional)

We are working with a single and pretty big pdf file. We want to load it in a loop and replace some images (there are some other operations as well, but it looks like only image insertions cause memory leaks). This causes memory to quickly start filling-up. It looks like some resources are not being properly released when the document is closed.

Regards,
Daniel

[JorjMcKie]

I'll have a look int this. Some preliminary comments.

• to some extent, this is normal, intended behaviour: MuPDF maintains its own caching mechanism for large objects types like images and fonts. It will keep these in memory even after close of the document.
• PyMuPDF has some functions to overrule this and force emptying the cache: fitz.TOOLS.store_shrink(percent) which releases a percentage of that cache.

[daniel-kukiela]

First of all - thank you for the really quick answer.

I tried fitz.TOOLS.store_shrink(100) in this toy example and it mostly works, so I thought it's going to be a solution for me. There is still some memory increase, but it is small enough that it might be not a big problem anymore.

I tried this with my actual code (added it after the self.document.close() in this case) and it did not seem to work. After some inspection, it looks like there is something more. This is how I am replacing images:

    def replace_pdf_image(self, image, new_image_bytes):
        self.document[image['page_number']].addRedactAnnot(image['position'])
        self.document[image['page_number']].apply_redactions()
        self.document[image['page_number']].insertImage(image['position'], stream=new_image_bytes)

I commented out the last line to discover that the issue is still there to some extent and it looks like this redaction also takes memory which it does not release.
Is there anything I can do in this case?

I tried to replicate this with the toy example:

import fitz
import cv2
import numpy
import io
import numpy as np

# Infinitive loop
while True:
    # Empty document for this example
    with fitz.open() as doc:
        # Let's also instert 100 empty pages
        for i in range(100):
            doc.insertPage(i)
            page = doc[i]

            # Create image rectangle
            image_rectangle = fitz.Rect(0, 0, 10, 10)

            # Example image - 1000x1000 px zaved as png into byte stream
            image = np.zeros((500, 500, 3))
            _, image_numpy_bytes = cv2.imencode(".png", image, [cv2.IMWRITE_PNG_COMPRESSION, 9])
            image_bytes = io.BytesIO(image_numpy_bytes)

            # Inserting image - test with this commented out to see no memory leaks, then uncomment and check again
            page.insertImage(image_rectangle, stream=image_bytes)

            # Replace image using redaction
            page.addRedactAnnot((0, 0, 10, 10))
            page.apply_redactions()
            page.insertImage((0, 0, 10, 10), stream=image_bytes)

    fitz.TOOLS.store_shrink(100)

but it does not look like the effect is the same. Maybe because I'm replacing a newly added image here and I'm replacing actual images with my other code? I'm not sure. What I know for sure is that when I commented out inserting images in my actual code, memory was still rising (even with fitz.TOOLS.store_shrink(100)), and when I add skipping adding and applying redactions - memory stops rising.

Regards,
Daniel

[JorjMcKie]

Independent from my previous post, you did find a serious issue!
So thank you for submitting this.

In previous releases, before inserting an image, MuPDF checked if there already exists the same in the PDF. This worked as an internally maintained MD5 table of the image binaries.
This has gone apparently - to my unpleasant surprise.
The consequence is, that repeated insertions of an image add up to the memory used. Not a memory leak in the strict sense of the word, but still unpleasant enough.
So I made some changes in my own code which log the images inserted by PyMuPDF using a similar mechanism as it used to be present in MuPDF.

Here is the script, a test image and the results log:
test1.zip

Are you always inserting / replacing with the same new image? Then my change should be an improvement for you.
It looks like if you use pixmaps for insertion, the situation in general is somewhat better. Using streams seems the worst alternative in any case.

If you have an urgent situation, I can create a pre-release wheel which you can pick up from the PyMuPDF-Wheels repo after I drop you a note.

Note on apply_redactions()

Try to use apply_redactions(images=fitz.PDF_REDACT_IMAGE_REMOVE). This removes the old image w/o trying anything more sophisticated.

[daniel-kukiela]

I am doing 2 things:

1. Loading images, modifying them in the fly, and replacing back into the pdf
2. Creating a couple of images on the fly and inserting

These images are assumed to be unique per iteration. Iteration is as follows: load the pdf file, replace images, insert new, save under another name, repeat (with the same pdf file).
So for me personally it would be ideal to purge the whole cache as I do not need to keep newly generated images in the memory once I place them back in the pdf. I might, however, benefit from a cache of the original images from the pdf file that I load and use each iteration, but I solved it in another way - I have a "pre" script which extracts them from the file and saves as a NumPy array dump, so this works also across script runs. So yes, an ability to clear caches would be ideal in my case I think.

I have to go for a few hours now, but I'll check what you asked for above right after I come back. Thank you!

Regards,
Daniel

[daniel-kukiela]

I'm back home and testing.
apply_redactions(images=fitz.PDF_REDACT_IMAGE_REMOVE) works for me ideally. Memory usage does not rise. I can say even more - when I look at the top (in Ubuntu), it was reporting 1.6-1.7GB of mamory usage after a single pass and now it's about 660MB so this also solves another issue (or maybe not issue, but something I would have to deal with). (System alone takes about 230MB when idling).

Thank you very much for your quick responses and support. I actually have a bit urgent situation, but you provided workarounds which work. I'll be happy to test further code updates if you'll have any.

Regards,
Daniel

[JorjMcKie]

happy to hear this and thanks for the feedback
please do not hesitate to come back with whatever may come across

[JorjMcKie]

New version with these changes being uploaded.

[daniel-kukiela]

Hey!

I made a mistake. I left a return in my image replacement function, like:

    def replace_pdf_image(self, image, new_image_bytes):
        return
        self.document[image['page_number']].addRedactAnnot(image['position'])
        self.document[image['page_number']].apply_redactions(images=fitz.PDF_REDACT_IMAGE_REMOVE)
        self.document[image['page_number']].insertImage(image['position'], stream=new_image_bytes)

I did not realize it up until now when I wanted to check something only to discover that the images are not replaced. After removing this return... my main issue is still there and memory usage raises constantly with redactions (also what I wrote about significantly lower memory usage now is invalid as well).
Is there anything else that I can try?

Regards,
Daniel

[daniel-kukiela]

I moved all of the code related to pdf file editing into a separate function, and I'm running it using Process from multiprocessing inside the loop, and this works. So I have a working workaround for now.

Regards,
Daniel

[JorjMcKie]

Hi, thanks for the update.
When I tested this I saw that using pixmaps instead of streams or files also had a beneficial effect on memory and speed.
This is because these objects already contain all information (width, height, alpha) which must otherwise be determined internally - a process that involves generating temporary ipixmaps.
So maybe this can some efficiency to your case, too.