Webcam input keeps failing - OS X

[ghost]

Hello, I'm trying to load a webcam using JavaCV (which appears to be OpenCV with Java bindings - please correct me if I'm wrong, or on the wrong repo) but after 2 to 3 minutes of continuously grabbing images off of the webcam we get an exception:

java.lang.RuntimeException: Unknown exception.
      at org.bytedeco.javacpp.opencv_videoio$VideoCapture.grab(Native Method)
      at org.bytedeco.javacpp.OpenCVFrameGrabber.grab(OpenCVFrameGrabber.java:244)
      at edu.wpi.grip.core.sources.GrabberService.runOneGrab(GrabberService.java:73)
      at edu.wpi.grip.core.sources.GrabberService.run(GrabberService.java:64)
      at com.google.common.util.concurrent.AbstractExecutionThreadService$1$2.run(AbstractExecutionThreadService.java:60)
      at com.google.util.concurrent.Callables$3.run(Callables.java:100)
      at java.lang.Thread.run(Thread.java:745)

Our GrabberService code (lines 59 to 100):

protected void run() throws GrabberServiceException {
   final OpenCVFrameConverter.ToMat convertToMat = new OpenCVFrameConverter.ToMat();
   final Stopwatch stopwatch = Stopwatch.createStarted();

   while(super.isRunning()) {
        runOneGrab(convertToMat, stopwatch);
   }
}

final void runOneGrab(final OpenCVFrameConverter.ToMat convertToMat, final Stopwatch stopwatch) throws GrabberServiceException {
   final Frame videoFrame;
   try {
      videoFrame = frameGrabber.grab();
   } catch(FrameGrabber.Exception ex) {
      throw new GrabberServiceException("Failed to grab image", ex);
   }

   final opencv_core.Mat frameMat = convertToMat.convert(videoFrame);

   if(frameMat == null || frameMat.isNull()) {
      throw new GrabberServiceException("Returned a null frame Mat");
   }

   if(frameMat.empty()) {
      throw new GrabberServiceException("Returned an empty Mat");
   }

   updater.copyNewMat(frameMat);

   stopwatch.stop();
   final long elapsedTime = stopwatch.elapsed(TimeUnit.MILLISECONDS);
   stopwatch.reset();
   stopwatch.start();
   if(elapsedTime != 0) {
      updater.setFrameRate(IntMath.divide(1000,Math.toIntExact(elapsedTime),RoundingMode.DOWN));
   }

   updater.updatesComplete();
   exceptionClearedCallback.run();
}

Is there something on our side that needs to be changed or does it look like a JavaCV issue?

[saudet]

Try FFmpeg instead. OpenCV is known to have various issues grabbing from cameras.

[saudet]

I haven't tested it, but according to this page, the following should work:

FrameGrabber grabber = new FFmpegFrameGrabber("");
grabber.setFormat("avfoundation");
grabber.start();

[ghost]

Besides also needing to add that to Gradle (v3.0.2-1.2, rest of the configuration should look like this compile group: 'org.bytedeco.javacpp-presets', name: 'ffmpeg', version: '3.0.2-1.2', classifier: os). However, after starting that up, I can't seem to get the webcams to load in - not even if I replace "" with "default" (which is what I found from FFmpeg that it should be fore OS X). Using the phrase "default" instead of "" actually causes Java to crash and even wants my Mac to send a bug report to Apple.

Is there any other solution - OpenCV works for Linux and VideoInputFrameGrabber works for Windows, I guess we just need a solution for OS X.

[saudet]

FFmpeg 3.1.1 is the latest version. Maybe you could try that and see?

[ghost]

I tried looking into FFmpeg 3.1.1 and that seemed to return some weird errors with grade. However, I started looking into it some more and found that the code that we have appears to be using a device number to identify the camera - which worked very well as far as OpenCVFrameGrabber was concerned. However, FFmpegFrameGrabber isn't configured to the webcam as the device number - so our current system is out of the question. I ended up attempting a StringBuilder system, which while that gets around the issue it won't be able to connect to the webcam built into the Macbook Pro while running macOS and device number '1' resolves to the monitor on macOS. (The entire desktop was displayed.)

The dependency stuff I used for FFmpeg 3.1.1 was: compile group: 'com.nextbreakpoint', name: 'com.nextbreakpoint.ffmpeg4java', version: '3.1.1-1.0', classifier: os which seemed to be unresolvable by gradle due to conflicts which concerned how the developers published that to maven.

Any ideas? Thanks for the help thus far 😄

[saudet]

If you want to use FFmpeg4Java, please post your questions there, not here, thank you.

[ghost]

I didn't see ffmpeg version 3.1.1 connected to bytedeco while looking at maven. Is there somewhere else that I should be looking for the source for Gradle instead of FFmpeg4Java?

[saudet]

Build from source.