Upgrade to Ffmpeg 3.x will cause FfmpegFrameRecorder record physical memory OOM

[gordon-rawe]

I've been recently using Ffmpeg to encode captured preview video on android platform, and the project address is as below:
https://github.com/gordon-rawe/Record-Ffmpeg-Android.git
When I upgrade the javacv version to 1.2 or above which means using the dependency of 3.x above version ffmpeg, the Ffmpeg will cause physical memory OOM, I have got some clues, it's cause by the recordImage function, seems the memories allocated are not released inside it, maybe cause by the newer api, but I tries to change the recordImage function, try to use av_frame_free and av_xxx functions to free unused variables,it didn't work, did you experts tried on these new APIs as they are marked deprecated? expecting your reply^_^

[saudet]

Do you get the same behavior with Java SE or does this happen on Android only?

[gordon-rawe]

Sorry to reply so late, I just tried this on android platform because which is my target platform, and I didn't try it on the SE yet! and I will try it later and tell you the result.

[saudet]

I don't think it will solve your issue, but I got rid of the deprecated calls in the commit above.

[v4ekmarev]

After upgrading from version 1.1 to version 1.3.1 get an
OD/dalvikvm: GC_EXPLICIT freed 5K, 33% free 23133K/34268K, paused 6ms+7ms, total 69ms
D/dalvikvm: GC_EXPLICIT freed 387K, 33% free 23183K/34268K, paused 18ms+4ms, total 98ms
D/dalvikvm: GC_EXPLICIT freed 13K, 33% free 23170K/34268K, paused 6ms+13ms, total 107ms
D/dalvikvm: GC_EXPLICIT freed <1K, 33% free 23170K/34268K, paused 7ms+17ms, total 119ms
D/dalvikvm: GC_EXPLICIT freed <1K, 33% free 23170K/34268K, paused 8ms+9ms, total 89ms
D/dalvikvm: GC_EXPLICIT freed <1K, 33% free 23170K/34268K, paused 7ms+9ms, total 100ms
D/dalvikvm: GC_EXPLICIT freed <1K, 33% free 23170K/34268K, paused 6ms+15ms, total 117ms
D/dalvikvm: GC_EXPLICIT freed <1K, 33% free 23170K/34268K, paused 7ms+10ms, total 110ms
D/dalvikvm: GC_EXPLICIT freed <1K, 33% free 23170K/34268K, paused 11ms+8ms, total 85ms
D/dalvikvm: GC_EXPLICIT freed <1K, 33% free 23170K/34268K, paused 6ms+7ms, total 97ms
D/dalvikvm: GC_EXPLICIT freed <1K, 33% free 23170K/34268K, paused 3ms+12ms, total 105ms
D/AndroidRuntime: Shutting down VM
W/dalvikvm: threadid=1: thread exiting with uncaught exception (group=0x41d36730)
E/AndroidRuntime: FATAL EXCEPTION: main
java.lang.OutOfMemoryError: Physical memory usage is too high: physicalBytes = 128M > maxPhysicalBytes = 128M
at org.bytedeco.javacpp.Pointer.deallocator(Pointer.java:562)
at org.bytedeco.javacpp.Pointer.init(Pointer.java:121)
at org.bytedeco.javacpp.BytePointer.allocateArray(Native Method)
at org.bytedeco.javacpp.BytePointer.(BytePointer.java:82)
at org.bytedeco.javacv.FFmpegFrameRecorder.recordImage(FFmpegFrameRecorder.java:849)
at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:827)
at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:820)
at JavaCVmediaSystem.writeVideoFrameIfRecording(JavaCVmediaSystem.java:232)

[saudet]

Could you guys call System.setProperty("org.bytedeco.javacpp.logger.debug", "true") during init and let me know what you get in the log with that?

[v4ekmarev]

https://gist.github.com/v4ekmarev/bdba931e511b5eecd5d643cc8b5e4cd1

[saudet]

@v4ekmarev Thanks! Not seeing anything wrong though.

Could someone try the latest code from the master branch and let me know if the issue is still happening?

[v4ekmarev]

I could try but I do not know how to build the library from source =)

[saudet]

It's also possible to just use the source code as part of your project...

[alexhutnik]

Trying to build the project but I don't think I have all the prereqs on my machine. Seems like, for instance, flycapture isn't downloaded to the downloads dir during build.

[saudet]

You don't need to build the presets, just JavaCV.

[alexhutnik]

Yes but the latest from your master branch depends on some SNAPSHOT artifacts that are not in Maven Central yet. So right now I am build OpenCV 3.2 in the presets, changing the JavaCV POM to use 3.2, and then building JavaCV.

[saudet]

We can backport the changes to a release version of JavaCV, that's much easier.

[alexhutnik]

Ok I modified the POM as follows:

diff --git a/pom.xml b/pom.xml
index 66b2150..61b1e90 100644
--- a/pom.xml
+++ b/pom.xml
@@ -61,7 +61,7 @@
   <properties>
     <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
     <maven.build.timestamp.format>yyyyMMddhhmm</maven.build.timestamp.format>
-    <javacpp.version>${project.version}</javacpp.version>
+    <javacpp.version>1.3</javacpp.version>
   </properties>
 
   <dependencies>
@@ -74,7 +74,7 @@
     <dependency>
       <groupId>org.bytedeco.javacpp-presets</groupId>
       <artifactId>opencv</artifactId>
-      <version>3.2.0-${javacpp.version}</version>
+      <version>3.1.0-${javacpp.version}</version>
     </dependency>
     <dependency>
       <groupId>org.bytedeco.javacpp-presets</groupId>
@@ -89,7 +89,7 @@
     <dependency>
       <groupId>org.bytedeco.javacpp-presets</groupId>
       <artifactId>libdc1394</artifactId>
-      <version>2.2.5-${javacpp.version}</version>
+      <version>2.2.4-${javacpp.version}</version>
     </dependency>
     <dependency>
       <groupId>org.bytedeco.javacpp-presets</groupId>

And it fails to compile due to missing symbols in PS3EyeFrameGrabber. Are there other modification I should make locally to test out the latest?

[saudet]

Do you have any problems building https://github.com/bytedeco/javacv/tree/1.3.1 ?

[alexhutnik]

Yes same issue. I've even cleared out my local .m2 repo directory to make sure I don't have anything half-baked in there. Compile error is this:

[ERROR] /home/user/tools/javacv/src/main/java/org/bytedeco/javacv/PS3EyeFrameGrabber.java:[311,83] cannot find symbol
[ERROR] symbol:   variable CLCamera
[ERROR] location: class org.bytedeco.javacv.PS3EyeFrameGrabber

[saudet]

Ah yes that's not available on Maven. You can erase that one.

[alexhutnik]

Since I'm using javacv-platform in my project, I'm not exactly sure how to build the chain

javacv-platform (1.3.1) -> javacv (1.3.2-SNAPSHOT) -> javacpp-presets.opencv (3.1.0-1.3 or 3.2.0-1.3.1-SNAPSHOT) -> javacpp-presets (1.3 or 1.3.1-SNAPSHOT)

[saudet]

Like I said, don't do that. You don't need to do that...

[saudet]

I've released JavaCV 1.3.2, which doesn't call deprecated functions anymore. If you still have issues with that, it's probably due to JavaCPP, not FFmpeg or JavaCV. Set the "org.bytedeco.javacpp.maxphysicalbytes" system property to "0" during initialization to get the old behavior back.

[saudet]

I think I've fixed all the memory leaks occurring in FFmpegFrameGrabber and FFmpegFrameRecorder.
Please give it a try with the snapshots: http://bytedeco.org/builds/