The sound at the end could not be recorded

[iwanglian]

I use the simple video example ,and set the time to 15s ,and about from 12s , no sound would be record into the video.

[BradLarson]

Try this now. I repaired some code related to audio recording, so that may have fixed this as well. I just recorded a video, and the audio was present throughout the entire movie.

[iwanglian]

I see, you add CVOpenGLESTextureCacheFlush when finish, but the example crashed.Then I comment this code ,and again, then end of sound go away.

dyld: Symbol not found: _CVOpenGLESTextureCacheFlush

Referenced from: /var/mobile/Applications/ADD3A9B3-576E-4AD0-96E2-3B43BFFB6430/SimpleVideoFilter.app/SimpleVideoFilter
Expected in: /System/Library/Frameworks/CoreVideo.framework/CoreVideo

[iwanglian]

CV_EXPORT void CVOpenGLESTextureCacheFlush(CVOpenGLESTextureCacheRef textureCache, CVOptionFlags options) __OSX_AVAILABLE_STARTING(__MAC_NA,__IPHONE_5_0);

[BradLarson]

Yes, the CVOpenGLESTextureCacheFlush() was an error I left in while trying to fix another issue yesterday. The latest code should have that removed.

Given that this crashed for you, I assume you are running this on an iOS 4.x device? What device and what iOS version are you seeing this missing audio on?

[iwanglian]

iPhone4 ios4.3.1

[rohamak]

also on iPhone4 iOS 5.1 always last 5 to 7 seconds of video is muted!!

[phil-m]

also getting missing audio for the last couple of seconds on an iPhone4 running iOS 5.0.1.
Is the audio side getting stopped/ignored before the video has a chance to stop? Will be testing on some other devices over the next couple of days to see if it happens on other hardware/OS combos & report back (looks like everyone here is having the issue on just iphone4?)

[rohamak]

I noticed something, when an inexpensive filter like grayscale is used the sound is okay all the way to the end, actually grayscale was the only filter I noticed works fine also I have not tested them all, possibly other filters have higher process which don't let the audio queue be processes concurrently on iPhone 4 hardware.

[shtefane]

Hi! Thank you Brad for your work!
I tried on iphone 4 iOS 5.1.1 with AVCaptureSessionPreset640x480 sound is record ok but with AVCaptureSessionPreset1280x720 last few seconds audio is muted...
Any suggestion !?

[valvoline]

Hi, i'm experiencing the same problem here (iPhone4S / iPhone4 with 5.1).

If i use an inexpensive filter, the audio goes well until the end. However, if I insert any other CPU intensive filter (like tiltShift, vignette and go on) the audio stop some seconds before the end of the video.

Is there any patch for this?

thank you

[valvoline]

Hi,

After investigating more in deep the problem, i found that it seems to be closely related to 1920x1080preset (and/or to CPU intensive blocks).

I managed some tests with two filters chained together with a FilterGroup (a ToneColorFilter and a VignetteFilter).

With the CaptureSession configured with a 1920x1080 output, both the iPhone4S and the iPhone4, output a video with the audio track truncated at the end.

With the CaptureSession configured with at 1280x720 output, iPhone4S output a correct video and audio track. The iPhone4, instead, cannot handle the output, giving the audio truncation problem.

[valvoline]

Hi,

After investigating more in deep the problem, i found that it seems to be closely related to 1920x1080preset (and/or to CPU intensive blocks).

I managed some tests with two filters chained together with a FilterGroup (a ToneColorFilter and a VignetteFilter).

With the CaptureSession configured with a 1920x1080 output, both the iPhone4S and the iPhone4, output a video with the audio track truncated at the end.

With the CaptureSession configured with at 1280x720 output, iPhone4S output a correct video and audio track. The iPhone4, instead, cannot handle the output, giving the audio truncation problem.

I tried all the available CapturePresets (1920x1080, 1280x720, 640x480), but with the iPhone4 the audio is truncated at the end in each test.

Hope this can help to diagnose the problem. Maybe, it's related to CPU intensive operations. This's my cent.

[ClovisMarsupial]

Same here. Is there any option or workaround ?

[valvoline]

Hi Clovis,

I've worked to a workaround for a while, but with poor results.

All that I can tell you is try to reduce the CPU/GPU load for your filter / filter chain. In my personal project I have a chain that looks like the following:

(Contrast) -> (Saturation) -> (Brightness) -> (Pixellate)

I reduced my filter chain, working on merging two of them (Saturation+Brightness). Since they are simple manipulations of RGB texture maps, I managed to integrate them in a all-in-one solution.

WIth this new configuration:

(MyCustomFilterShader) -> (Pixellate)

I can record movie at 1920x1080 without any issue on an iPhone4S and at 640x680 on iPhone4.

I know. It sounds like a scary "workaround", but this is all that i can tell you, yet.

As far as I know this is a known limitation of old A4 based chipset (like the iPhone4). They lack of many features, like texture caching and so on, that are really important for live video processing.

Hope to hear some news from Brad.

c.

On Aug 3, 2012, at 10:03 PM, ClovisMarsupial reply@reply.github.com wrote:

Same here. Is there any option or workaround ?

---

Reply to this email directly or view it on GitHub:
#67 (comment)

[BradLarson]

The texture caches are actually present on all older devices, you just need iOS 5.0 to be able to use them. The A5 devices do have much larger maximum texture sizes (4096 vs. 2048) and significantly GPUs for shader processing, which results in vastly different performance characteristics.

Honestly, I don't work much with the movie reading and recording, so someone else might need to dig into this.

[valvoline]

I just downloaded the rosywriter Apple example. Seems that it is a good starting point to understand how and why the audio track fails during recording.

At a glance, I found that the RosyWriter example uses two different queues to record both audio and video. Also they are configured as SERIAL QUEUES.

Moreover, it uses a different approach for previewing, using the built-in AVFoundation feature.

I don't know if they are significant differences for this bug, but I will give them a try.

c.

On 04/ago/2012, at 23:41, Brad Larsonreply@reply.github.com wrote:

The texture caches are actually present on all older devices, you just need iOS 5.0 to be able to use them. The A5 devices do have much larger maximum texture sizes (4096 vs. 2048) and significantly GPUs for shader processing, which results in vastly different performance characteristics.

Honestly, I don't work much with the movie reading and recording, so someone else might need to dig into this.

---

Reply to this email directly or view it on GitHub:
#67 (comment)

[BradLarson]

I've now reworked the audio input to run on its own queue, which should prevent dropping of audio samples. It causes a few frames of video to now be dropped under heavy load, but I believe this should prevent audio from being cut off.

[valvoline]

Hi Brad,

Thanks for you reply. I just tried your fix. The audio seems working fine now. However something nasty happened to the VideoCamera outputImageOrientation and the setInputOrientation methods.

Now I can not handle correctly the image aspect ratio and orientation change anymore!

Have you same hints about? Are you aware of this?

Thanks

c.

On Aug 14, 2012, at 7:45 PM, Brad Larson notifications@github.com wrote:

I've now reworked the audio input to run on its own queue, which should prevent dropping of audio samples. It causes a few frames of video to now be dropped under heavy load, but I believe this should prevent audio from being cut off.

—
Reply to this email directly or view it on GitHub.

[valvoline]

Ok,

I modified the GPUImageVideoCamera in order to return ALWAYS outputRotation = kGPUImageNoRotation;

and then I start recording using the method

       [movieWriter startRecordingInOrientation:CGAffineTransformMakeRotation(rotationValue)];

the rotationValue is changed according to the actual orientation.

With this setup the actual build runs fine, both on iPhone4 (1280x720, iOS6b3) and iPhone4S (1920x1080, iOS511). And I don't experience any audio issues.

Without the above changes, in my setuo the GPUImageVideoCamera return always a wrong aspect ratio.

On Aug 14, 2012, at 7:45 PM, Brad Larson notifications@github.com wrote:

I've now reworked the audio input to run on its own queue, which should prevent dropping of audio samples. It causes a few frames of video to now be dropped under heavy load, but I believe this should prevent audio from being cut off.

—
Reply to this email directly or view it on GitHub.

[BradLarson]

@valvoline Try with the latest code in the repository. I tested this, and the output image orientation works perfectly in the SimpleVideoFilter example for all orientations I tried (remembering to adjust the output movie size to match the image size, of course). Perhaps there was a race condition there before, but this seems to work fine now.

[valvoline]

@BradLarson ,

Sorry for lag, but i was out for vacation. I just tried the 9addf9d and it seems to work nicely without any issues.

Rotating is now working as expected and Audio is encoded well without holes at the end. Both on iPhone4 and iPhone4S.

I think that this bug can now be closed.