Pixmap.tobytes, from Bytes to numpy.ndarray

[p1-dta]

Hi,

I work with Bytes object from Pixmap.tobytes and I want to convert it to a jpeg format. Actually, I use io.BytesIO, then PIL.Image.open to save the output in the desired format (.save(output_buffer, format="jpeg", quality=jpeg_quality)) then .getvalue() on the result.

I use .tobytes("ppm")

Using PIL might be inefficient, so I want to use OpenCV with cv2. But when I tried to create an image (numpy.ndarray I think) that cv2 can handle. I have errors like

>       im = imdecode(raw_bytes, IMREAD_COLOR)
E       cv2.error: OpenCV(4.5.3) :-1: error: (-5:Bad argument) in function 'imdecode'
E       > Overload resolution failed:
E       >  - buf is not a numpy array, neither a scalar
E       >  - Expected Ptr<cv::UMat> for argument 'buf'

Maybe I can use imread from cv2 but it read from a file, is there a good solution to pass the bytes pymupdf create to this function ?

Do you have any suggestion? Did I miss something ?

Thanks

[p1-dta]

I found a solution. I'll expose it in an edit.

Here is my workaround:

I must convert the bytes object to a numpy.bytearray.
then create a numpy.array from the bytearray with numpy.frombuffer.
Then imdecode from this numpy array and IMREAD_COLOR.

cv2_image = imdecode(numpy.frombuffer(bytearray(raw_bytes), dtype=numpy.uint8), IMREAD_COLOR)

[JorjMcKie]

The simplest thing to do is Pixmap.pil_tobytes(...). This offers you using all Pillow arguments, e.g. creating a bytes object in JPEG format is as simple as pix.pil_tobytes("JPEG"). There you could add all the goodies like "optimize" and what not.

[p1-dta]

Thanks

[paopjian]

only using the numpy.frombuffer is more efficient:
numpy.frombuffer(buffer=pix.samples, dtype=np.uint8).reshape((pix.height, pix.width, 3))
cost 1/10 time of
cv2_image = imdecode(numpy.frombuffer(bytearray(raw_bytes), dtype=numpy.uint8), IMREAD_COLOR)
you take too much covert on data style.

from https://blog.csdn.net/qq_36998298/article/details/124177620

[canklot]

Instead of manually specifying the 3 dimensions you can pass -1 as a last argument and let the numpy find the correct dimensions for you. Can be useful in gray or black and white images.

pix = np.frombuffer(buffer=pix.samples, dtype=np.uint8).reshape((pix.height, pix.width, -1))

[JorjMcKie]

I think .frombuffer also accepts memoryview objects, not only bytes-likes. Alongside pix.samples, which is a bytes copy of the internal area in the pixmap, there also exist pix.samples_mv which is a memoryview to that internal area (without copying), and pix.samples_ptr which is a Python pointer to that area.

So whenever a consumer of pixmaps can digest memoryviews or even Python pointers, they should definitely choose these alternatives.
Unfortunately, Pillow has no such option, but I know that newer versions of Qt support Python pointers - see PyMuPDF documentation here. Using the samples pointer instead of the bytes copy for Qt image creation can be up to 1000 times faster!

[paopjian]

amazing!

[JorjMcKie]

Thank you, but there is typo: only 12.4 times faster.

[JorjMcKie]

@paopjian - Additional comment: the memoryview alternative is independent from the pixmap size and remains around 1.5 microseconds even for very large pixmaps. Whereas the duration creating from pix.samples increases massively with its size. See the following measurement:

In [8]: %timeit np.frombuffer(pix.samples_mv, dtype=np.uint8).reshape((pix.height, pix.width, 3))
1.5 µs ± 14.7 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)
In [9]: %timeit np.frombuffer(pix.samples   , dtype=np.uint8).reshape((pix.height, pix.width, 3))
1.86 ms ± 2.85 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)
In [10]: pix
Out[10]: Pixmap(DeviceRGB, IRect(0, 0, 2272, 1704), 0)
In [11]:

Using pix.samples in this case was 1240 (!!!) times slower. Understandable, if considering that its size was about 11.6 MB.

[korotaS]

Hi!
Although np.frombuffer(pix.samples_mv...) is indeed much faster than np.frombuffer(pix.samples...) and consumes much less memory, I have faced the problem with RGB/BGR - sometimes the image created by np.frombuffer(pix.samples_mv...) was RGB and other times it was BGR, so be careful!

[jonashaag]

NOTE: If you use samples_mv, you may run into buffer reuse (?) problems. I'm assuming this is because the NumPy array is a read-only (non-copy) view on the internal pymupdf buffer that can be reused for other pages' samples_mv. So, be sure to copy the buffer or the image:

np.frombuffer(pix.samples_mv, dtype=np.uint8).reshape((pix.height, pix.width, 3)).copy()

Or

Image.fromarray(np.frombuffer(pix.samples_mv, dtype=np.uint8).reshape((pix.height, pix.width, 3))).copy()