60FPS Video Support

[PkmX]

Investigate ways to support 60fps video recording on the Nexus 5.

[Naparajito16]

Hello, without the IMX_179_60FPS libraries how will you implement 60fps? Also, Media_profiles.xml can be edited but for that you will have to have root perms.
Thanks.

[andriy2]

@Naparajito16 can you help me with editing Media_profiles.xml correct?
I have root, but I don't know, what I must change in that file

[Naparajito16]

@andriy2 Sure. Its pretty easy. But the app should support it from itself. If we depend on libs like the LG G2, then i must say it would be impossible. I can help you out, its very easy.

[andriy2]

@Naparajito16 and I could record videos such those?
720p - 60fps - https://www.youtube.com/watch?v=W6zI5CAciHM
480p - 120fps - https://www.youtube.com/watch?v=L9X-nVgRrHg

[PkmX]

Good news everyone! I've got 60fps recording working on the Nexus 5! Sample video: https://www.youtube.com/watch?v=T6D1Qu7Q23o

However, this is not working without caveats:

• This requires patching a library on the device, which requires root. Patching the library will break most (all?) other camera apps and render the device unable to capture images at resolutions larger than 1632x1224.
• The output video is actually a crop of the upper-left quarter of the sensor, rather than using the full sensor at a lower resolution. This makes composition very awkward.
• Since this is a 1/4 crop of the sensor, the maximum supported video size is 1600x1200 (or 1280x720 for 16:9).

I would like to specially thank Martin Wawro for his help in enabling 60fps support. His original effort required using a native app that directly uses Qualcomm's libraries to record the video, and I managed to adapt his modifications to run on L Camera directly without extensive modifications to Android's framework. 120fps support is also possible, but I'm not sure if it is practical as it will result in a very heavily-cropped region and the maximum resolution would be only 800x600.

It's a very hacky solution at the moment, and I will see if I can come up with a better and more robust solution in the next few days. Stay tuned!

[PkmX]

Another sample video: https://www.youtube.com/watch?v=kTL3FfGV2k4

[andreash97]

great news :) @PkmX

[ivandisi]

Very nice news !!!

[mohammad-sajjad11]

Why not contact google developers about it ? @PkmX

[PkmX]

@msamii Good point. Maybe I should open a ticket on Android's issue tracker.

Ideally, pixel binning or skipping mechanisms should be used rather than cropping the sensor output, but this entirely depends on whether Qualcomm will change their libraries to support these operations. Without access to the source code and data sheets of the imaging sensor, it is nearly impossible for us developers to enable those modes (and we don't even know if it's supported by the hardware).

This test at least shows that it is possible for the hardware to operate at faster than 30 fps. I was originally worried that the sensor could only do an image readout every 1/30 second, and if that was the case >30 fps recording would be basically impossible.

[PkmX]

Support for 60fps recording has been pushed to the repository.

[BigSo]

Hello, I would like to know if the application is still current.
You are still working on the application or not?

: I Thank You for the works

[kshepitzki]

Hi
I tried running lcamera but it gives many compilation errors and I don't speak scala.
I need help with a difficult android camera performance issue.
I am developing a camera application were I need to display the camera preview without conceivable latency, so when the user looks at the world and the preview together he shouldn’t conceive a time gap between them.
Every frame needs to be scaled, flipped, rotated and overlaid with a small image before being displayed but all these manipulations add less than 1ms to the computations so no problem there.
The minimal latency I measure on my app, on some example projects and on the built-in camera is 120ms (tested on Note4, Galaxy 4 etc) which is definitely noticeable.
I must reduce it to 60ms. The best frame rate I managed to get is a stable 31 fps. I tried both the old Camera API and Camera2 API both with SurfaceView and TextureView.
The app needs to run only on a single phone model which is based on the Snapdragon 615 SoC and the OmniVision 13850 camera running OS 5.0.2. I need to display the preview on a 1080x608 area of the display. So there is no shortage of CPU power, as can be seen from the attached Systrace the processor is idle most of the time:
https://drive.google.com/open?id=0B780awz-gp4URDFPXzBNRVMwa0k
https://drive.google.com/open?id=0B780awz-gp4UVEE2NGRmbEJPdlU
GPU profiling shows me that rendering is well below the 16ms mark. The 13850 can provide 30, 60 and even 120 fps.
If I understand the problem correctly, the limit comes from the camera VSync being capped at 32ms (to a maximum of 30 fps) by the Android system (maybe in the kernel drivers) or from the 13850 configuration.
According to Google (https://source.android.com/devices/graphics/architecture.html#SurfaceFlinger) Android is using triple buffering so at 32ms that’s 96ms which with some overheads can explains the 120ms.
I assume that if I could increase camera preview fps to 60 my latency will decrease to 60ms.
There are many discussions how to achieve 60 fps for drawing graphics on the display but I can’t find much about getting 60 fps from the camera preview.
Is my assumption correct?
How do I decrease the camera preview latency to 60ms? (It’s a dedicated device so I am allowed to make changes to Linux and drivers).
Thanks