Raw capture with resizer is broken

[soren121]

I'm attempting to write a program that captures a raw stream from the video port and passes it to OpenCV, but the program fails when trying to capture the stream. I know it's possible, because before I switched to using Python, I was using a C++ class that could do this. At least, I think that's what it does.

It's definitely not an OpenCV error, as the program dies before any OpenCV functions are called.

Here's the error:

Traceback (most recent call last):
    File "main.py", line 122, in <module>
        main()
    File "main.py", line 117, in main
        frame = cam.read()
    File "/home/pi/2014-Vision/src/picam.py", line 16, in read
        self.cam.capture(stream, 'rgba', True, (320, 240))
    File "/usr/lib/python2.7/dist-packages/picamera/camera.py", line 859, in capture
        if not encoder.wait(30):
    File "/usr/lib/python2.7/dist-packages/picamera/encoders.py", line 326, in wait
        raise self.exception
AssertionError

And the code:

from __future__ import division

import picamera
import cv2
import numpy as np
import io
import time

class PiCam:

    def __init__(self):
        self.cam = picamera.PiCamera()

    def read(self):
        stream = open('image.data', 'wb')
        self.cam.capture(stream, 'bgra', True, (320, 240))
        image = np.fromfile(stream, dtype = np.uint8)
        return image

cam = PiCam()
cv2.namedWindow("Camera Image", cv2.WINDOW_AUTOSIZE)
while True:
    cv2.imshow("Camera Image", cam.read())
    cv2.waitKey(1)
    return

[waveform80]

That's weird - raw capture over the video port is included in the test suite, and passes (more or less) happily (there's some issue with 100x100 captures that I don't understand yet). Unfortunately, my camera's currently busy chewing through yet another test for #40 (which is unsurprisingly taking some time to debug and fix - given the random nature of it showing up). Could you possibly test your code without the resizer? Something like the following:

from __future__ import division

import picamera
import cv2
import numpy as np
import io
import time

class PiCam:

    def __init__(self):
        self.cam = picamera.PiCamera()
        self.cam.resolution = (320, 240)

    def read(self):
        stream = open('image.data', 'wb')
        self.cam.capture(stream, 'bgra', use_video_port=True)
        stream.seek(0)
        image = np.fromfile(stream, dtype = np.uint8)
        return image

cam = PiCam()
cv2.namedWindow("Camera Image", cv2.WINDOW_AUTOSIZE)
while True:
    cv2.imshow("Camera Image", cam.read())
    cv2.waitKey(1)
    return

In theory it shouldn't make any difference as the raw encoder was changed in 1.0 to use the resizer to handle the conversion from YUV to RGBA/BGRA/etc. (so that we don't have to manipulate the camera's own ports and break video recording, and because the video-splitter doesn't seem to like doing the conversion itself). In other words, all raw captures invoke the resizer but if no resize resolution is given, the resizer is set to the same resolution as the camera itself.

Still, the test suite doesn't actually include a test for raw captures over the video port including a resizer with a different resolution to the camera, so I'm wondering if that's the cause.

[soren121]

That seems to work. The size of the output stream is correct, according to your RGB recipe math (320 * 240 * 4=307200.) Getting OpenCV to read it is another story. Thanks.

[waveform80]

Glad you got something working in the end - I'll try and find some time to look into this, but I doubt it'll be in time to make the release I'm planning for later today

[waveform80]

Doh! Found the cause of this issue - a stupid error on my part. It's a trivial fix so it'll make it into tonight's release after all. Amended the title as the bug would actually cause any raw capture with a resize parameter (set to something other than the parent's resolution) to fail.