Capturing CameraStream raises error "No data, did you record anything?"

[frostieDE]

I am trying to capture 100 snapshots programatically from my webcam but somehow, the ImageRecorder does not record any image. However, when capturing the images using the dedicated widget button, it works like charm.

This is my code:

First code block (which simply shows the user the webcam input)

from ipywebrtc import CameraStream, ImageRecorder
from time import sleep

camera = CameraStream.facing_user(audio=False)
camera

Second code block (which should capture 100 shots from the camera):

recorder = ImageRecorder(stream=camera)
recorder.recording = True

for i in range(1,100):
    sleep(0.1)
    recorder.save("webcam/" + str(i) + ".png")

Executing the last block, the following error appears:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-9-77bec100e9f4> in <module>
      4 for i in range(1,100):
      5     sleep(0.1)
----> 6     recorder.save("webcam/" + str(i) + ".png")

c:\program files (x86)\microsoft visual studio\shared\python37_64\lib\site-packages\ipywebrtc\webrtc.py in save(self, filename)
    397             filename += '.' + self.format
    398         if len(self.image.value) == 0:
--> 399             raise ValueError('No data, did you record anything?')
    400         with open(filename, 'wb') as f:
    401             f.write(self.image.value)

ValueError: No data, did you record anything?

What am I missing here?

[martinRenou]

recorder.recording = True will send to the frontend an event that will say "please take a snapshot". So this is a kind of asynchronous statement. That must be the reason why you cannot directly save it the line after in Python. I suppose you would need to execute your loop in another cell.

Also, the recorder.recording = True will be set back to False after a snapshot has been taken. So you'll be able to take only one with your code.

I have an example here https://github.com/QuantStack/quantstack-talks/blob/master/2018-11-14-PyParis-widgets/notebooks/6.ipywebrtc.ipynb that does approximately what you want to achieve. The important part of the code is this one:

out = Image()
stop_process = False

def process_image(_):
    if stop_process:
        return
    im_in = PIL.Image.open(io.BytesIO(image_recorder.image.value))
    im_array = np.array(im_in)[...,:3]
    im_array_edges = adapt_rgb(each_channel)(sobel)(im_array)
    im_array_edges = ((1-im_array_edges) * 255).astype(np.uint8)
    im_array_edges[im_array_edges < 250] = 0
    im_out = PIL.Image.fromarray(im_array_edges)
    f = io.BytesIO()
    im_out.save(f, format='png')
    out.value = f.getvalue()
    image_recorder.recording = True

image_recorder.image.observe(process_image, names=['value'])

HBox([out, image_recorder])

I observe the value of the recorder, everytime it changes it means that we took a new snapshot and process_image is called, then at the end I do image_recorder.recording = True for recording a new snapshot. It's like an infinite callback loop

[martinRenou]

I suppose you would need something like:

recorder = ImageRecorder(stream=camera)
i = 0

def save_html(_):
    if i > 100:
        return
    recorder.save("webcam/" + str(i) + ".png")
    i = i + 1
    recorder.recording = True

recorder.image.observe(save_html, names=['value'])
recorder.recording = True

[frostieDE]

Thanks for the clarification on how the library works :-)

But something still seems to be missing here as the callback does not seems to be triggered. I tried the example notebook you mentioned above and it seems to working (in your example the user needs to click on the capture button in order to start capturing, but even that does not work for me):

Tested on latest Firefox and Chrome on Windows.

from ipywebrtc import CameraStream, ImageRecorder
from ipywidgets import Output, VBox

camera = CameraStream.facing_user(audio=False)
camera

recorder = ImageRecorder(stream=camera)
recorder

out = Output()

numImage = 0
numImages = 100

def savePicture(_):
    if i >= numImages:
        return
    
    out.append_stdout("Image " + numImage + " captured \n")
    recorder.save("webcam/" + str(i) + ".png")
    numImage = numImage + 1
    recorder.recording = True
    
recorder.image.observe(savePicture, names=['value'])

VBox([recorder, out])

[martinRenou]

Is it the full code? i seems to not be defined, I suppose

if i >= numImages:

should be

if numImage >= numImages:

[frostieDE]

I feel so silly 🤦‍♂ But unfortunately, it does not seem to solve the issue :-(

Even commenting out the non-relevant parts does not seem to fix it:

out = Output()

numImage = 0
numImages = 100

def savePicture(_):
    #if numImage >= numImages:
    #    return
    
    #out.append_stdout("Image " + numImage + " captured \n")
    recorder.save("webcam/" + str(i) + ".png")
    #numImage = numImage + 1
    recorder.recording = True
    
recorder.image.observe(savePicture, names=['value'])
VBox([recorder, out])

[martinRenou]

You also have this line:

recorder.save("webcam/" + str(i) + ".png")

That should be

recorder.save("webcam/" + str(numImage) + ".png")

[martinRenou]

The annoying thing is that if there is an error in the callback you won't see it... And it fails silently

[martinRenou]

If you try something like:

def savePicture(_):
    with out:
        if i >= numImages:
            return
    
        out.append_stdout("Image " + numImage + " captured \n")
        recorder.save("webcam/" + str(i) + ".png")
        numImage = numImage + 1
        recorder.recording = True

You should see errors in the Output widget I think

[frostieDE]

Very nice, now I see what's going wrong: the variable scope does not seem to inherit as I expected.

UnboundLocalError                         Traceback (most recent call last)
<ipython-input-5-21253cfd001e> in savePicture(_)
      6 def savePicture(_):
      7     with out:
----> 8         if numImage >= numImages:
      9             return
     10 

UnboundLocalError: local variable 'numImage' referenced before assignment

Code:

out = Output()

numImage = 0
numImages = 100

def savePicture(_):
    with out:
        if numImage >= numImages:
            return

        out.append_stdout("Image " + numImage + " captured \n")
        recorder.save("webcam/" + str(numImage) + ".png")
        numImage = numImage + 1
        recorder.recording = True
    
recorder.image.observe(savePicture, names=['value'])
VBox([recorder, out])

[martinRenou]

I'm not sure I get why...

This could make it work, but I don't see why it is needed:

def savePicture(_):
    with out:
        global numImage
        global numImages

        [...]

[frostieDE]

It finally works 🎉 In case someone wants to do the same as I want, this is the code:

from ipywebrtc import CameraStream, ImageRecorder
from ipywidgets import Output, VBox

camera = CameraStream.facing_user(audio=False)
camera

recorder = ImageRecorder(stream=camera)
out = Output()

numImage = 0
numImages = 100

def savePicture(_):
    with out:
        global numImage
        global numImages
        if numImage >= numImages:
            return

        out.append_stdout("Image " + str(numImage) + " captured \n")
        recorder.save("webcam/" + str(numImage) + ".png")
        numImage = numImage + 1
        recorder.recording = True
    
recorder.image.observe(savePicture, names=['value'])
VBox([recorder, out])