Problem with blocking method waitForNewFrame()

[0x54]

waitForNewFrame() is blocking indefinitely. I'm using the code in Python 2.7, in a multiprocessing class.
On the same machine, the multiframe_listener example is working like a charm.

any idea?

[r9y9]

I guess you forgot to call listener.release(frames)? See

pylibfreenect2/examples/multiframe_listener.py

Line 83 in edc7964

listener.release(frames)

.

[0x54]

Thanks for your answer! Yes, you have right. But it keeps blocking. No frame grabbed... It's strange couse if I put the code in the "run" method it does not work, but if I exec it in the class init it works.

[r9y9]

Can you paste a code so that I can reproduce the issue?

[0x54]

This code isn't working:

`from multiprocessing import Process
from pylibfreenect2 import Freenect2, SyncMultiFrameListener
from pylibfreenect2 import FrameType, Registration, Frame
from pylibfreenect2 import createConsoleLogger, setGlobalLogger
from pylibfreenect2 import LoggerLevel
try:
from pylibfreenect2 import OpenGLPacketPipeline
pipeline = OpenGLPacketPipeline()
except:
from pylibfreenect2 import CpuPacketPipeline
pipeline = CpuPacketPipeline()

class Processor(Process):

def __init__(self):
    super(Processor, self).__init__()

    self.fn = Freenect2()
    num_devices = self.fn.enumerateDevices()

    if num_devices == 0:
        print("No device connected!")
        sys.exit(1)

    self.serial = self.fn.getDeviceSerialNumber(0)
    self.device = self.fn.openDevice(self.serial, pipeline=pipeline)
    self.listener = SyncMultiFrameListener(FrameType.Color | FrameType.Ir | FrameType.Depth)

    # Register listeners
    self.device.setColorFrameListener(self.listener)
    self.device.setIrAndDepthFrameListener(self.listener)
    self.device.start()

    self.registration = Registration(self.device.getIrCameraParams(),
                        self.device.getColorCameraParams())
    self.undistorted = Frame(512, 424, 4)
    self.registered = Frame(512, 424, 4)

    # Optinal parameters for registration
    # set True if you need
    need_bigdepth = False
    need_color_depth_map = False

    self.bigdepth = Frame(1920, 1082, 4) if need_bigdepth else None
    self.color_depth_map = np.zeros((424, 512),  np.int32).ravel() \
        if need_color_depth_map else None

def run(self):
    while True:
        if self.listener.hasNewFrame():
            frames = self.listener.waitForNewFrame()
            print frames
            color = frames["color"]
            ir = frames["ir"]
            depth = frames["depth"]
            self.registration.apply(color, depth, self.undistorted, self.registered,
                       bigdepth=self.bigdepth,
                       color_depth_map=self.color_depth_map)
            cv2.imshow("color", cv2.resize(color.asarray(),
                               (int(1920), int(1080))))
            self.listener.release(frames)

            key = cv2.waitKey(delay=1)
            if key == ord('q'):
                break

p = Processor()
p.start()`

On the other hand, this one is working:

`from multiprocessing import Process
import numpy as np
import cv2
from pylibfreenect2 import Freenect2, SyncMultiFrameListener
from pylibfreenect2 import FrameType, Registration, Frame
from pylibfreenect2 import createConsoleLogger, setGlobalLogger
from pylibfreenect2 import LoggerLevel

class Processor(Process):

def __init__(self):
    super(Processor, self).__init__()
    try:
        from pylibfreenect2 import OpenGLPacketPipeline
        pipeline = OpenGLPacketPipeline()
    except:
        from pylibfreenect2 import CpuPacketPipeline
        pipeline = CpuPacketPipeline()
    self.fn = Freenect2()
    num_devices = self.fn.enumerateDevices()

    if num_devices == 0:
        print("No device connected!")
        sys.exit(1)

    self.serial = self.fn.getDeviceSerialNumber(0)
    self.device = self.fn.openDevice(self.serial, pipeline=pipeline)
    self.listener = SyncMultiFrameListener(FrameType.Color | FrameType.Ir | FrameType.Depth)

    # Register listeners
    self.device.setColorFrameListener(self.listener)
    self.device.setIrAndDepthFrameListener(self.listener)
    self.device.start()

    self.registration = Registration(self.device.getIrCameraParams(),
                        self.device.getColorCameraParams())
    self.undistorted = Frame(512, 424, 4)
    self.registered = Frame(512, 424, 4)

    # Optinal parameters for registration
    # set True if you need
    need_bigdepth = False
    need_color_depth_map = False

    self.bigdepth = Frame(1920, 1080, 4) if need_bigdepth else None
    self.color_depth_map = np.zeros((424, 512),  np.int32).ravel() \
        if need_color_depth_map else None

    while True:
        if self.listener.hasNewFrame():
            frames = self.listener.waitForNewFrame()
            print frames
            color = frames["color"]
            ir = frames["ir"]
            depth = frames["depth"]
            self.registration.apply(color, depth, self.undistorted, self.registered,
                       bigdepth=self.bigdepth,
                       color_depth_map=self.color_depth_map)
            cv2.imshow("color", cv2.resize(color.asarray(),
                               (int(1920), int(1080))))
            self.listener.release(frames)

            key = cv2.waitKey(delay=1)
            if key == ord('q'):
                break


def run(self):
    pass

p = Processor()
p.start()`

[r9y9]

umm, I cannot reproduce the issue.. I used the similar code above: https://gist.github.com/r9y9/c64d4500283f6c33aa150d1e89806af8 (removed multiprocessing related things, since I don't have it).

[r9y9]

It would be helpful if you can paste a complete log message.

[0x54]

There isn't any error message, it just doesn't work: it wait indefinitely on waitForNewFrame(). The strange behavior appears only when subclassing multiprocessing. I have just tried subclassing object and it's working. Any idea?

Also, I have noted much more cpu usage when using the library code in a class. How can it be?
hasNewFrame() increases the cpu usage...

[r9y9]

Can you share the complete test codes? I don't have the multiprocessing package. Also could you tell me if https://gist.github.com/r9y9/c64d4500283f6c33aa150d1e89806af8 works or not?

[0x54]

The code at https://gist.github.com/r9y9/c64d4500283f6c33aa150d1e89806af8 works without any problem.
The problem appears subclassing Process frommultiprocessing, here the complete code: https://gist.github.com/0x54/9664f12d61918d027136c403e018d1c3
When I run it, there isn't any error message and the console output seems to be ok:

..but unfortunately there isn't any window showing the video from the Kinect, unlike your code.

[0x54]

Thanks for your help!!!

[kunwu522]

Hi @0x54 ，I got same problem here. Did you find any way to solve this?
Thanks.

[jmarkow]

Same problem here.

[VitusMa]

I am also having this problem. Does anyone know a solution?

[jmarkow]

In order to run other computations while grabbing frames I put the frame-grabbing related code in the main thread of my script. Looks like this breaks when you run on child processes via multiprocessing I'm afraid.