Frame Tearing with a Kiyo Pro at 1080p

[ThatWileyGuy]

I'm running this on an RPi 4 with a Kiyo Pro at 1080p. Here's my config:

[server]
listen =
port = 8000
[/dev/video0]
width = 1920
height = 1080
fps = 30

# Device capture format (default: H264)
# H264, MJPGH264, YUYV, MJPG, JPEG
capture_format = NV12

# Decoder M2M device (default: disabled)
# To decode the stream to a compatible format with the encoder (eg MJPG -> NV12 -> H264)
# decoder = /dev/video10

# Encoder M2M device (default: disabled)
# To encode the stream to H264 (eg YUYV -> H264 or MJPG -> NV12 -> H264)
encoder = /dev/video11

# Auto Sleep mode (default: yes)
# Sleep the camera when no one is watching the stream
auto_sleep = yes

# Controls

# you can set any V4L2 control too, list them with the -l option
h264_profile = High
h264_level = 4.2
h264_i_frame_period = 15

uvcx_h264_i_frame_period = 1000
uvcx_h264_profile = High

# Advanced (change only if you know, what you are doing)

# Buffer memory configurations (MMAP or DMABUF)
# Sometimes contigous memory is not large enough for hardwares and tuning needed

# default: DMABUF if encoder or decoder else MMAP
# capture_memory = DMABUF

# decoder_memory = MMAP-DMABUF
# encoder_memory = MMAP-MMAP

# Input format for the decoder (default: MJPG if capture_format == JPEG else capture_format)
# If the capture_format isn't supported by the decoder directly,
# but it can decode it with another format eg (capture_format = JPEG, decoder_input_format = MJPG)
# decoder_input_format = MJPG

# Input format for the encoder (default: NV12 if decoder else capture_format)
# encoder_input_format = NV12

My problem is that randomly and particularly when there's motion (like waving my hand in front of the camera), the video feed will "tear" where below some line the image in my browser will be corrupted. It's almost like the frame data is getting truncated, but I don't see how that could make sense for h264 encoded video?

I'm going to try a different camera to be sure and see if I can upload a video that shows the issue if it still reproes. Have you heard of or seen anything similar before?

[ThatWileyGuy]

Tried to get this working with a non-pro Kiyo I had sitting around and couldn't get MJPG -> NV12 -> H264 to work (seems like the RPI can't decode 1080p MJPG?), but the hardware H264 encoder on the camera was flawless.

Flipped the Kiyo Pro over to hardware H264 and I still see corruption, but now it's clearly dropped frames post-encoder rather than corrupted frames getting fed into the encoder (instead of a tear line, I see smeared motion and such). My guess is that the USB3 controller on the RPi is having trouble and dropping frames between the camera and the RPi?

I modified the read loop to look like this:

            try:
                if not camera.sleeping:
                    camera.request_key_frame()
                cameraSleeper.add_client()
                mp4_writer = MP4Writer(self.wfile, config.width(), config.height(), config.timescale(), h264parser.sps, h264parser.pps)
                desired_frame_time = 1.0 / 30.0
                frame_start = time()
                while True:
                    nalus, frame_secs, frame_usecs = h264parser.read_frame()
                    read_done = time()
                    mp4_writer.add_frame(nalus, frame_secs, frame_usecs)

                    frame_end = time()
                    frame_time = frame_end - frame_start
                    if frame_time > 2 * desired_frame_time:
                        print("long frame: ", frame_time / desired_frame_time, " read: ", (read_done - frame_start) / desired_frame_time, " write: ", (frame_end - read_done) / desired_frame_time)
                    frame_start = frame_end
            except Exception as e:
                cameraSleeper.remove_client()
                self.log_message(f'Removed streaming client {self.client_address} {e}')

... and I see a steady stream of long frames that are 2-4 times the desired frame time, with nearly all the time spent waiting on the frame from the camera.

[soyersoyer]

The Kiyo Pro firmware has issues. Sometimes it doesn't set the USB EOF bit and it causes delays on Linux. I opened tickets at Razer, but they only care about Windows. I tried with a Pi on USB2, it worked with YUYV in lower resolution/lower framerate. I don't have a Pi with free USB3 port yet.

[ThatWileyGuy]

Hilariously, there's actually an available firmware update for my camera (1.00 -> 1.05) which claims to have Fixed an issue where the image lags or freezes intermittently. Unfortunately, the updater utility tells me "To proceed with the update, please contact Razer Support" and from searching around, it sounds like Razer shipped an early hardware revision that can't take the firmware update and has to be replaced.

I'm torn between making Razer send me a replacement and just never buying any of their products again, but either way, this doesn't seem to be anything solvable in fmp4streamer.

Thanks!

[ThatWileyGuy]

Just wanted to confirm here in case anyone else uses one of these cameras - on the most recent firmware (1.05), this issue no longer occurs and my Kiyo Pro works fine with fmp4streamer.
If you have this problem, follow the instructions at https://mysupport.razer.com/app/answers/detail/a_id/4582
If the firmware updater refuses to update your camera, post in Razer's forums or message a member of their support team and they'll open an RMA to replace your camera.