capture_sequence() and capture_continuous() only capture RAW Bayer data on first frame

[laser101]

According to the documentation...

The bayer parameter of the capture() method causes the raw Bayer data recorded by the camera’s sensor to be output as part of the image meta-data.

I assume (maybe wrongly) that the bayer parameter should also be supported by the capture_sequence() and capture_continuous() methods.

However, tests show that the raw Bayer data is only added to the header of the first JPEG frame captured when using capture_sequence() and capture_continuous() methods, and not subsequent frames.

For example, when capturing 3 frames using the capture() method...

camera.capture('bayer0.jpeg', format='jpeg', bayer=True)
camera.capture('bayer1.jpeg', format='jpeg', bayer=True)
camera.capture('bayer2.jpeg', format='jpeg', bayer=True)

We generate 3 JPEG encoded images approximately 7MB in size, with the Bayer data packed in the header just as we would expect.

ls -l
-rw-r--r-- 1 pi pi 7224575 Mar  1 12:32 bayer0.jpeg
-rw-r--r-- 1 pi pi 7244684 Mar  1 12:32 bayer1.jpeg
-rw-r--r-- 1 pi pi 7229215 Mar  1 12:32 bayer2.jpeg

However, if we try to do the equivalent thing using capture_sequence() or capture_continuous() methods then the resulting file sizes indicate that only the first image has the Bayer data in the header (and indeed I have confirmed this by inspecting the data).

ls -l
-rw-r--r-- 1 pi pi 7206216 Mar  1 12:37 bayer0.jpeg
-rw-r--r-- 1 pi pi  821918 Mar  1 12:37 bayer1.jpeg
-rw-r--r-- 1 pi pi  815960 Mar  1 12:37 bayer2.jpeg

A demonstration of the problem follows. Simply uncomment the appropriate function call at the bottom of the script.

import time
import picamera
import io

bayer_mode = True
dir = '/home/pi/camera/bayer_bug'
num_of_images = 3


def filename_gen(num_of_frames):
    """
    Filename generator
    """
    frame = 0
    while frame < num_of_frames:
        yield '%s/bayer%d.jpeg' % (dir, frame)
        frame += 1



def capture_single_filenames(num_of_images):
    """
    Use camera.capture() method using filename as 1st argument
    """
    print('Using camera.capture() with filename...')
    stream = io.BytesIO()
    with picamera.PiCamera() as camera:
        camera.start_preview()
        # Let the camera warm up for a couple of seconds
        time.sleep(2)

        gen = filename_gen(num_of_images)
        start = time.time()
        for count in range(num_of_images):
            # Capture the image, including the Bayer data
            camera.capture(next(gen), format='jpeg', bayer=bayer_mode)
            # print('Captured image %d' % count)
        elapsed = time.time() - start
        print('Captured %d frames at %.2ffps' % (num_of_images, num_of_images / elapsed) )


def capture_single_stream(num_of_images):
    """
    Use camera.capture() method using stream as 1st argument
    """
    print('Using camera.capture() with stream...')
    stream = io.BytesIO()
    with picamera.PiCamera() as camera:
        camera.start_preview()
        # Let the camera warm up for a couple of seconds
        time.sleep(2)
        stream = io.BytesIO()

        start = time.time()
        for count in range(num_of_images):
            # Capture the image, including the Bayer data
            camera.capture(stream, format='jpeg', bayer=bayer_mode)
            # print('Captured image %d' % count)
            # Get number of bytes in the stream
            num_of_bytes = stream.tell()
            # Rewind the stream to start
            stream.seek(0)
            # Save stream contents to file
            filename = '/home/pi/camera/bayer_bug/bayer%d.jpeg' % count
            with open(filename, 'wb') as f:
                f.write( stream.read( num_of_bytes ) )
            # Empty the stream
            stream.seek(0)
            stream.truncate()
        elapsed = time.time() - start
        print('Captured %d frames at %.2ffps' % (num_of_images, num_of_images / elapsed) )


def capture_sequence_filenames(num_of_images):
    """
    Use camera.capture_sequence() method
    """
    print('Using camera.capture_sequence() with filename generator...')
    with picamera.PiCamera() as camera:
        camera.start_preview()
        # Let the camera warm up for a couple of seconds
        time.sleep(2)

        start = time.time()
        camera.capture_sequence( filename_gen(num_of_images), format='jpeg', bayer=bayer_mode )
        elapsed = time.time() - start
        print('Captured %d frames at %.2ffps' % (num_of_images, num_of_images / elapsed) )


def capture_continuous_filenames(num_of_images):
    """
    Use camera.capture_continuous() method using filename as 1st argument
    """
    print('Using camera.capture_continuous() with filename...')
    with picamera.PiCamera() as camera:
        camera.start_preview()
        # Let the camera warm up for a couple of seconds
        time.sleep(2)

        start = time.time()
        for count, filename in enumerate( camera.capture_continuous( '/home/pi/camera/bayer_bug/bayer{counter}.jpeg', format='jpeg', bayer=bayer_mode ) ):
            # print('Captured image %d' % count)
            if count == (num_of_images-1):
                break
        elapsed = time.time() - start
        print('Captured %d frames at %.2ffps' % (num_of_images, num_of_images / elapsed) )


def capture_continuous_stream(num_of_images):
    """
    Use camera.capture_continuous() method using stream as 1st argument
    """
    print('Using camera.capture_continuous() with stream...')
    stream = io.BytesIO()
    with picamera.PiCamera() as camera:
        camera.start_preview()
        # Let the camera warm up for a couple of seconds
        time.sleep(2)
        stream = io.BytesIO()

        start = time.time()
        for count, foo in enumerate( camera.capture_continuous( stream, format='jpeg', bayer=bayer_mode ) ):
            # print('Captured image %d' % count)
            # Get number of bytes in the stream
            num_of_bytes = stream.tell()
            # Rewind the stream to start
            stream.seek(0)
            # Save stream contents to file
            filename = '/home/pi/camera/bayer_bug/bayer%d.jpeg' % count
            with open(filename, 'wb') as f:
                f.write( stream.read( num_of_bytes ) )
            # Empty the stream
            stream.seek(0)
            stream.truncate()
            if count == (num_of_images-1):
                break
        elapsed = time.time() - start
        print('Captured %d frames at %.2ffps' % (num_of_images, num_of_images / elapsed) )



# capture_single_filenames(num_of_images)
#capture_single_stream(num_of_images)
capture_sequence_filenames(num_of_images)
#capture_continuous_stream(num_of_images)
#capture_continuous_filenames(num_of_images)

[waveform80]

Interesting - it might be that the request for the bayer data needs to be made on every encoding start. If that's all it takes, it's a simple fix which I can get into 1.11.

[6by9]

Yup, it needs to be set before setting the capturing parameter every time. The original implementation passed in a path/filename, so you didn't want it overwriting files.

[laser101]

It would be great to see some progress on this in 1.11. This issue is still causing me some pain.

[waveform80]

PRs welcome! Unfortunately at the moment I've got bigger issues with 1.11 that need sorting first (segfaults in data callbacks which I've spent two days trying to track - fruitlessly so far).