mp.Image can't deal with numpy array views

[JC3]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

Yes

OS Platform and Distribution

macos

Mobile device if the issue happens on mobile device

No response

Browser and version if the issue happens on browser

No response

Programming Language and version

python

MediaPipe version

0.10.9

Bazel version

No response

Solution

pose

Android Studio, NDK, SDK versions (if issue is related to building in Android environment)

No response

Xcode & Tulsi version (if issue is related to building for iOS)

No response

Describe the actual behavior

When presented with a numpy array view as its data parameter, the mp.Image constructor fails.

Describe the expected behaviour

When presented with any numpy array type, the mp.Image constructor succeeds.

Standalone code/steps you may have used to try to get what you need

# Retrieve video from OpenCV.
vcap = cv2.VideoCapture(0)
ok, frame = vcap.read()

# Crop to left half of image.
frame = frame[:,:int(frame.shape[1]/2),:]

# Construct mp.Image
mp_frame = mp.Image(image_format=mp.ImageFormat.SRGB, data=frame)

Other info / Complete Logs

Traceback (most recent call last):
  File "server.py", line 194, in run
    mp_frame = mp.Image(image_format=mp.ImageFormat.SRGB, data=frame)
TypeError: __init__(): incompatible constructor arguments. The following argument types are supported:
    1. mediapipe.python._framework_bindings.image.Image(image_format: mediapipe::ImageFormat_Format, data: numpy.ndarray[numpy.uint8])
    2. mediapipe.python._framework_bindings.image.Image(image_format: mediapipe::ImageFormat_Format, data: numpy.ndarray[numpy.uint16])
    3. mediapipe.python._framework_bindings.image.Image(image_format: mediapipe::ImageFormat_Format, data: numpy.ndarray[numpy.float32])

[JC3]

Workaround is:

frame = np.array(frame)

Prior to constructing an mp.Image.

[kuaashish]

Hi @JC3,

This behavior is known to us. There is no plan to extend the support to the numpy views. As you have already investigated, a simple preprocessing step is to convert the view back to a numpy array.

Thank you!!

[kuaashish]

Hi @JC3,

Can you please let us know the current status of the issue? If it's no longer an issue on your end, we can go ahead and close it, marking it as resolved.

Thank you!!

[JC3]

It's no big deal since the work around is easy. Thanks!

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No