Frame event never trigger

[wishfulthinkerme]

Hey,
I am using RPI 3b, and camera OV5647.

It seems that .on('frame') is never triggered. I do not need a lot of fps, even 10 would be enough, but StillICamera is too slow, but I can't find any possibility to send to my server frames separately as JPG images with using StreamCamera.

I receive data with using videoStream.on('frame'), but these are not frames.

Any tips how can I achieve this?

Best Regards,
Mateusz

[gregnr]

Hi @wishfulthinkerme, thanks for the issue. Up until now, most testing as been on the v2 camera (IMX219), but I would like this library to fully support all camera versions, including v1 (OV5647) and HQ (IMX477). Once we get our hands on a OV5647, we will investigate this.

[darrylhodgins]

@wishfulthinkerme, I'm using the same hardware (but using takeImage() instead of on('frame')).

This is working for me; are you missing the MJPEG codec option?

const streamCamera = new StreamCamera({
    codec: Codec.MJPEG
});
await streamCamera.startCapture();
const image = await streamCamera.takeImage();

[gregnr]

@wishfulthinkerme Can you please confirm if @darrylhodgins's suggestion fixes your issue?

[wishfulthinkerme]

@gregnr As far as I remember it didn't solve the issue :(

[axwaxw]

I am having the same issue.

I am using Raspberry Pi 4 Model B Rev 1.1 and the new High Quality Camera. (I have added this pi model in my local version of the package - see here)

The streamCamera.takeImage() is working, but on('frame') never fires for me. I can see that streamCamera.takeImage() utilises once('frame') under the hood, so that's weird.

Using another package here, the on('frame') does work.

Code to reproduce this issue

const { StreamCamera, Codec } = require("pi-camera-connect");

const streamCamera = new StreamCamera({
  codec: Codec.MJPEG
});

const videoStream = streamCamera.createStream();

await streamCamera.startCapture();

videoStream.on("frame", data => console.log("New frame", data));

// NOTHING IN CONSOLE?

// Wait 5 seconds
await new Promise(resolve => setTimeout(() => resolve(), 5000));

await streamCamera.stopCapture();

[gregnr]

@axwaxw Thanks for the thorough details. I'm hoping to get my hands on a Pi 4 in the next couple of weeks and will try to see if I can reproduce your issue. With that said, I'm suspicious the issue is more likely related to the HQ Camera model generating a different JPEG signature, and not based on Pi model. If you are able to generate ~5 seconds of MJPEG data using your HQ Camera, I would be happy to parse it to see if a different JPEG signature is used.

The approach used by node-raspberry-pi-camera-native is interesting. You can see here they are not separating frames by JPEG signature, rather based on the size of the data coming back.

Unlike node-raspberry-pi-camera-native which creates C++ native bindings to stream camera data, pi-camera-connect streams camera data from a child process (raspivid). This means we are subject to the inter-process pipe buffer size set by the OS, and can't guarantee that a certain number of bytes should be expected back, or even that the final data event for a frame won't also contain data for the next frame.

JPEG signatures are probably still the most accurate method to parse frames, we just need to ensure we support every possible signature.

[axwaxw]

@gregnr
Thanks. As requested, MJPEG attached from the new high quality camera.
video-stream.zip

[gregnr]

@axwaxw Thanks for the sample. I took a look and your MJPEG video does have the same JPEG signature as the other cameras. With that said, I think I know why you're having issues (same for @wishfulthinkerme).

In your sample code above you are listening to an on('frame') event on the videoStream instead of the streamCamera. Try instead listening to the streamCamera.on('frame') event.

I can see how this can be confusing. As it is today, streamCamera.createStream() simply creates a vanilla stream.Readable with no additional functionality, while the streamCamera itself has the additional on('frame') event. We could change this to augment the stream.Readable with an additional on('frame') event if that's more intuitive. Another option is to forget streamCamera.createStream() altogether, and treat streamCamera itself as a stream.Readable with some extra functionality (like on('frame')). The latter option might be the most ergonomic.

Any opinions or feedback is welcome.

[axwaxw]

@gregn thanks for getting back on this. That makes sense. I think a couple of comments in other issues (the issue about client side) put me wrong. I am sure this syntax is fine - perhaps just needs another example in the docs?

[axwaxw]

@gregnr fyi the code below (which is mostly derived from this project - thanks @caseymcj) now works.

My next mission is to come up with a way to stream simultaneously to multiple clients connected at the same time...

const { StreamCamera, Codec } = require("pi-camera-connect");
const express = require('express')
const app = express()

app.listen(3000, () => console.log(`Listening on port 3000!`));

const runApp = async () => {
  const streamCamera = new StreamCamera({
    codec: Codec.MJPEG,
    width: 1280,
    height: 720,
    fps: 15,
  });

  await streamCamera.startCapture();

  app.get("/stream.mjpg", (req, res) => {

    res.writeHead(200, {
      'Cache-Control': 'no-store, no-cache, must-revalidate, pre-check=0, post-check=0, max-age=0',
      Pragma: 'no-cache',
      Connection: 'close',
      'Content-Type': 'multipart/x-mixed-replace; boundary=--myboundary'
    });

    console.log('Accepting connection: ' + req.hostname);
    let isReady = true;

    let frameHandler = (frameData) => {
      try {
        if (!isReady) {
          return;
        }
        isReady = false;
        console.log('Writing frame: ' + frameData.length);
        res.write('--myboundary\nContent-Type: image/jpg\nContent-length: ${frameData.length}\n\n');
        res.write(frameData, function () {
          isReady = true;
        });

      }
      catch (ex) {
        console.log('Unable to send frame: ' + ex);
      }
    }

    let frameEmitter = streamCamera.on('frame', frameHandler);

    req.on('close', () => {
      frameEmitter.removeListener('data', frameHandler);
      console.log('Connection terminated: ' + req.hostname);
    });
  })
}
runApp();

... and on the client (HTML):

<html>
  <body>
    <img src="http://192.168.10.61:3000/stream.mjpg">
  </body>
</html>

[gregnr]

@wishfulthinkerme I am closing this based on the assumption that videoStream.on('frame') was incorrectly being used instead of streamCamera.on('frame'). If this wasn't the case, feel free to reopen this.

@axwaxw Glad this is working for you now. Please feel free to open another issue if you need help with anything else unrelated to frame even triggering.