New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Playing audio #458

Open
pgodino opened this Issue Sep 5, 2012 · 48 comments

Comments

Projects
None yet
@pgodino

pgodino commented Sep 5, 2012

When I play a movie with audio, no audio is played. I checked the code and actually it seems that audio is considered only when writing to file.

@BradLarson

This comment has been minimized.

Show comment
Hide comment
@BradLarson

BradLarson Sep 5, 2012

Owner

You're right, there's nothing in the framework to support audio playback from movies right now. I don't do much work with movie playback myself, so I haven't spent any time looking into this.

I do welcome any contributions you or others might make to add this functionality, though.

Owner

BradLarson commented Sep 5, 2012

You're right, there's nothing in the framework to support audio playback from movies right now. I don't do much work with movie playback myself, so I haven't spent any time looking into this.

I do welcome any contributions you or others might make to add this functionality, though.

@pgodino

This comment has been minimized.

Show comment
Hide comment
@pgodino

pgodino Sep 6, 2012

I'm trying to implement audio playing but I'm stuck. Since AVAssetReader reads both audio and video I thought to implement audio playing in this way:

  • add to AVAssetReader also the audio track
  • in readNextAudioSampleFromOutput adding a call to a new method processAudioFrame after calling processAudioBuffer (that I think handles writing the audio component to a file)
  • in the new method using CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer to get the AudioBufferList which contains the audio samples
  • paying audio samples using RemoteIO audio units (I think that a low level audio playing is required in order to get opmimal performances) as described for example here: http://atastypixel.com/blog/using-remoteio-audio-unit/.

In this way I get only noise. I think that possible problems are (coding problems. I configured AVAssetReaderTrackOutput with this dictionary of options (in processAsset):

NSMutableDictionary *audioOutputSettings = [NSMutableDictionary dictionary];
[audioOutputSettings setObject:[NSNumber numberWithInt:kAudioFormatLinearPCM] forKey:AVFormatIDKey];
[audioOutputSettings setObject:[NSNumber numberWithInt:44100] forKey:AVSampleRateKey];
[audioOutputSettings setObject:[NSNumber numberWithInt:16] forKey:AVLinearPCMBitDepthKey];
[audioOutputSettings setObject:[NSNumber numberWithBool:NO] forKey:AVLinearPCMIsBigEndianKey];
[audioOutputSettings setObject:[NSNumber numberWithBool:NO] forKey:AVLinearPCMIsFloatKey];
[audioOutputSettings setObject:[NSNumber numberWithBool:NO] forKey:AVLinearPCMIsNonInterleaved];

    // This might need to be extended to handle movies with more than one audio track
    AVAssetTrack* audioTrack = [audioTracks objectAtIndex:0];
    readerAudioTrackOutput = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:audioTrack outputSettings:audioOutputSettings];
    [reader addOutput:readerAudioTrackOutput];

And in this way audio samples in the buffer I get should be signed int 16 bits.

The second problem I see is the synchronization between reader (the callback required by remoteIO audio unit) ad the writer of the audio samples. I'm working assuming that this is the problem building a buffer queue.

Some help from someone with some experience with playing the audio from an AVAssetReader could be very useful.

pgodino commented Sep 6, 2012

I'm trying to implement audio playing but I'm stuck. Since AVAssetReader reads both audio and video I thought to implement audio playing in this way:

  • add to AVAssetReader also the audio track
  • in readNextAudioSampleFromOutput adding a call to a new method processAudioFrame after calling processAudioBuffer (that I think handles writing the audio component to a file)
  • in the new method using CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer to get the AudioBufferList which contains the audio samples
  • paying audio samples using RemoteIO audio units (I think that a low level audio playing is required in order to get opmimal performances) as described for example here: http://atastypixel.com/blog/using-remoteio-audio-unit/.

In this way I get only noise. I think that possible problems are (coding problems. I configured AVAssetReaderTrackOutput with this dictionary of options (in processAsset):

NSMutableDictionary *audioOutputSettings = [NSMutableDictionary dictionary];
[audioOutputSettings setObject:[NSNumber numberWithInt:kAudioFormatLinearPCM] forKey:AVFormatIDKey];
[audioOutputSettings setObject:[NSNumber numberWithInt:44100] forKey:AVSampleRateKey];
[audioOutputSettings setObject:[NSNumber numberWithInt:16] forKey:AVLinearPCMBitDepthKey];
[audioOutputSettings setObject:[NSNumber numberWithBool:NO] forKey:AVLinearPCMIsBigEndianKey];
[audioOutputSettings setObject:[NSNumber numberWithBool:NO] forKey:AVLinearPCMIsFloatKey];
[audioOutputSettings setObject:[NSNumber numberWithBool:NO] forKey:AVLinearPCMIsNonInterleaved];

    // This might need to be extended to handle movies with more than one audio track
    AVAssetTrack* audioTrack = [audioTracks objectAtIndex:0];
    readerAudioTrackOutput = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:audioTrack outputSettings:audioOutputSettings];
    [reader addOutput:readerAudioTrackOutput];

And in this way audio samples in the buffer I get should be signed int 16 bits.

The second problem I see is the synchronization between reader (the callback required by remoteIO audio unit) ad the writer of the audio samples. I'm working assuming that this is the problem building a buffer queue.

Some help from someone with some experience with playing the audio from an AVAssetReader could be very useful.

@pgodino

This comment has been minimized.

Show comment
Hide comment
@pgodino

pgodino Sep 7, 2012

OK I'm almost there. I'm using https://github.com/michaeltyson/TPCircularBuffer for the consumer/producer buffer for AVAssetReader and RemoteIO audio unit.

I have still 2 problems:

  • I'm trying with an 8 secs video and audio processing is much faster that the video one. It seems that the audio processing is not realtime but faster. This has the effect that the circular buffer should be very big in order to store all the audio samples. And I get some acceleration of the audio at the end of the video. This means that it could be worse and worse as the video size increases.
  • I get some noise at the end of the of the audio queue but I think it could be related to the first problem.

I hope to be able to solve this problems next week in order to submit a pull request (but I don't know exactly how to do it, is there some documentations?)

pgodino commented Sep 7, 2012

OK I'm almost there. I'm using https://github.com/michaeltyson/TPCircularBuffer for the consumer/producer buffer for AVAssetReader and RemoteIO audio unit.

I have still 2 problems:

  • I'm trying with an 8 secs video and audio processing is much faster that the video one. It seems that the audio processing is not realtime but faster. This has the effect that the circular buffer should be very big in order to store all the audio samples. And I get some acceleration of the audio at the end of the video. This means that it could be worse and worse as the video size increases.
  • I get some noise at the end of the of the audio queue but I think it could be related to the first problem.

I hope to be able to solve this problems next week in order to submit a pull request (but I don't know exactly how to do it, is there some documentations?)

@pgodino

This comment has been minimized.

Show comment
Hide comment
@pgodino

pgodino Sep 27, 2012

Actually I wasn't able to solve the two problems I wrote before so I took a completely different direction (I had quite short sequences so I play the image sequence frame by frame especially in order to start and stop the "video" fast on iPad1).
Anyway if you wish I can share the code I wrote to play audio but I don't know how I can do it.

pgodino commented Sep 27, 2012

Actually I wasn't able to solve the two problems I wrote before so I took a completely different direction (I had quite short sequences so I play the image sequence frame by frame especially in order to start and stop the "video" fast on iPad1).
Anyway if you wish I can share the code I wrote to play audio but I don't know how I can do it.

@gordemota

This comment has been minimized.

Show comment
Hide comment
@gordemota

gordemota Sep 28, 2012

you can copy paste the audio code section and take out the rest.... and put it at https://gist.github.com/

however, i dont have short sequence since my video is 3-4 minutes at least

you can copy paste the audio code section and take out the rest.... and put it at https://gist.github.com/

however, i dont have short sequence since my video is 3-4 minutes at least

@pgodino

This comment has been minimized.

Show comment
Hide comment
@pgodino

pgodino Oct 2, 2012

Sorry if I did't answer before. I put the complete modified GPUImageMoview with the audio part. Here you have the link: https://gist.github.com/3819907

I hope it could be useful as a starting point.

pgodino commented Oct 2, 2012

Sorry if I did't answer before. I put the complete modified GPUImageMoview with the audio part. Here you have the link: https://gist.github.com/3819907

I hope it could be useful as a starting point.

@linktoming

This comment has been minimized.

Show comment
Hide comment
@linktoming

linktoming Oct 17, 2012

I also need this feature. I'm considering adopting a simpler work around. Play the audio of the origin file of the GPUImageMovie. Probable the sync between video and audio will be a problem since GPUImageMovie may not play at the original speed.

I also need this feature. I'm considering adopting a simpler work around. Play the audio of the origin file of the GPUImageMovie. Probable the sync between video and audio will be a problem since GPUImageMovie may not play at the original speed.

@scottmcdonnell

This comment has been minimized.

Show comment
Hide comment
@scottmcdonnell

scottmcdonnell Nov 29, 2012

Great library! I also need audio playback and sync to video.

I have also looked at Mo DeJong’s library AVAnimator which seems to have solved similar issues syncing a CAF file to the video images as they decode and adjusting the playback frame to stay in sync.

Do you think any of this code could be applied? I unfortunately cannot get my head around doing the mash up.

Great library! I also need audio playback and sync to video.

I have also looked at Mo DeJong’s library AVAnimator which seems to have solved similar issues syncing a CAF file to the video images as they decode and adjusting the playback frame to stay in sync.

Do you think any of this code could be applied? I unfortunately cannot get my head around doing the mash up.

@darcy-rayner

This comment has been minimized.

Show comment
Hide comment
@darcy-rayner

darcy-rayner Mar 7, 2013

There was an issue with the timestep of movie playback in GPUImageMovie. Movie's gradually loose time, which causes audio to desync. Anyway, I modified GPUImageMovie to support basic audio playback, (using an AVAudioPlayer). See here . Play a movie file with audio, and make sure to set the playSound property to true. The code tries to sync the movie playback to the audio. If the movie can't be decoded in realtime or better, than it will still probably fall out of sync. This code hasn't been thoroughly tested, so take it with a grain of salt.

There was an issue with the timestep of movie playback in GPUImageMovie. Movie's gradually loose time, which causes audio to desync. Anyway, I modified GPUImageMovie to support basic audio playback, (using an AVAudioPlayer). See here . Play a movie file with audio, and make sure to set the playSound property to true. The code tries to sync the movie playback to the audio. If the movie can't be decoded in realtime or better, than it will still probably fall out of sync. This code hasn't been thoroughly tested, so take it with a grain of salt.

@snofla

This comment has been minimized.

Show comment
Hide comment
@snofla

snofla Mar 12, 2013

@darcy-rayner playing audio during recording freezes up the video camera. any idea why?

snofla commented Mar 12, 2013

@darcy-rayner playing audio during recording freezes up the video camera. any idea why?

@darcy-rayner

This comment has been minimized.

Show comment
Hide comment
@darcy-rayner

darcy-rayner Mar 12, 2013

I've only tested it for playing back audio from a movie file. When playSound is turned on, the timestep of the video is tied to the audio track. My guess is it is trying to sync to an audio track with doesn't exist.

I've only tested it for playing back audio from a movie file. When playSound is turned on, the timestep of the video is tied to the audio track. My guess is it is trying to sync to an audio track with doesn't exist.

@snofla

This comment has been minimized.

Show comment
Hide comment
@snofla

snofla Mar 13, 2013

@darcy-rayner Fixed it It's just an audio session setup problem:

[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayback error:nil];
UInt32 enabled = 1;
AudioSessionSetProperty (kAudioSessionProperty_OverrideCategoryMixWithOthers, sizeof(enabled), &enabled);
[[AVAudioSession sharedInstance] setActive:YES error:nil];

snofla commented Mar 13, 2013

@darcy-rayner Fixed it It's just an audio session setup problem:

[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayback error:nil];
UInt32 enabled = 1;
AudioSessionSetProperty (kAudioSessionProperty_OverrideCategoryMixWithOthers, sizeof(enabled), &enabled);
[[AVAudioSession sharedInstance] setActive:YES error:nil];
@notedit

This comment has been minimized.

Show comment
Hide comment
@notedit

notedit May 11, 2013

maybe AudioQueue is the right choice. i know a little about the AudioQueue, does anyone who know AudioQueue want to work on this?

notedit commented May 11, 2013

maybe AudioQueue is the right choice. i know a little about the AudioQueue, does anyone who know AudioQueue want to work on this?

@JacobDev

This comment has been minimized.

Show comment
Hide comment
@JacobDev

JacobDev Jun 26, 2013

Hi. I need some help. I integrated the audio playback following the changes mentioned in the previous comments. I am facing a problem: audio playback is working fine in the simulator but when running my application on a device, it is not working. No error is logged, so I have no idea why this is happening. Has someone encountered the same issue?

Hi. I need some help. I integrated the audio playback following the changes mentioned in the previous comments. I am facing a problem: audio playback is working fine in the simulator but when running my application on a device, it is not working. No error is logged, so I have no idea why this is happening. Has someone encountered the same issue?

@davidgloves

This comment has been minimized.

Show comment
Hide comment
@davidgloves

davidgloves Jul 15, 2013

@snofla
Hi,I want to know where to add your code. I also have the problem. When adding a filter to a sample movie file, the audio goes normally while the video freezes. these are my codes:
movieFile.playSound = YES;
movieWriter.shouldPassthroughAudio = NO;
movieFile.audioEncodingTarget = movieWriter;
[movieFile enableSynchronizedEncodingUsingMovieWriter:movieWriter];
My purpose is processing a movie file with a filter. At beging the result has no sound, and the video has no sound too when processing. But now the video freezes. Thanks! Please help me out of this.

@snofla
Hi,I want to know where to add your code. I also have the problem. When adding a filter to a sample movie file, the audio goes normally while the video freezes. these are my codes:
movieFile.playSound = YES;
movieWriter.shouldPassthroughAudio = NO;
movieFile.audioEncodingTarget = movieWriter;
[movieFile enableSynchronizedEncodingUsingMovieWriter:movieWriter];
My purpose is processing a movie file with a filter. At beging the result has no sound, and the video has no sound too when processing. But now the video freezes. Thanks! Please help me out of this.

@jasaiher

This comment has been minimized.

Show comment
Hide comment
@jasaiher

jasaiher Jul 28, 2013

@darcy-rayner I try you GPUImageMovie with the last framework code. It´s working for the .m4v file in the example but I cant do for other video formats. Are you still working on this?

@darcy-rayner I try you GPUImageMovie with the last framework code. It´s working for the .m4v file in the example but I cant do for other video formats. Are you still working on this?

@davidgloves

This comment has been minimized.

Show comment
Hide comment
@davidgloves

davidgloves Jul 28, 2013

@jasaiher hey, it isn't m4v it's mp4. You can see the profile of the file, it's mp4 file. I think that you can change other file to the format of mp4.

@jasaiher hey, it isn't m4v it's mp4. You can see the profile of the file, it's mp4 file. I think that you can change other file to the format of mp4.

@eluzix

This comment has been minimized.

Show comment
Hide comment
@eluzix

eluzix Sep 3, 2013

as we encountered the same problem we decided to add audio support to GPUImageMovie using AudioUnits. You can grab our code at: https://github.com/pheed/GPUImage

To use link your project with AudioToolbox and CoreAudio frameworks than just set the playSound to YES on the GPUImageMovie.

few issues:

  1. we rewritten the playAtActualSpeed logic as the old one was out of sync once we added the audio.
  2. only one of playSound/audioEncodingTarget is supported.
  3. the audio player doesn't support pause/play

We are planning to issue a pull request sometime in the future after we test it further.

eluzix commented Sep 3, 2013

as we encountered the same problem we decided to add audio support to GPUImageMovie using AudioUnits. You can grab our code at: https://github.com/pheed/GPUImage

To use link your project with AudioToolbox and CoreAudio frameworks than just set the playSound to YES on the GPUImageMovie.

few issues:

  1. we rewritten the playAtActualSpeed logic as the old one was out of sync once we added the audio.
  2. only one of playSound/audioEncodingTarget is supported.
  3. the audio player doesn't support pause/play

We are planning to issue a pull request sometime in the future after we test it further.

@oltis

This comment has been minimized.

Show comment
Hide comment
@oltis

oltis Oct 10, 2013

Have you tested it? It is safe to use or there are some issues we should be aware of if we intend to use it?

Thanks

oltis commented Oct 10, 2013

Have you tested it? It is safe to use or there are some issues we should be aware of if we intend to use it?

Thanks

@eluzix

This comment has been minimized.

Show comment
Hide comment
@eluzix

eluzix Oct 10, 2013

@oltis we tested it relative to our needs and it seems to be working fine.

We did further changes to the GPUImage project to fit our needs but it shouldn't effect how it behaves under normal circumstances, if it does just grab the sources under GPUImage/Sources/Audio and GPUImageMovie.

eluzix commented Oct 10, 2013

@oltis we tested it relative to our needs and it seems to be working fine.

We did further changes to the GPUImage project to fit our needs but it shouldn't effect how it behaves under normal circumstances, if it does just grab the sources under GPUImage/Sources/Audio and GPUImageMovie.

@shruthimr

This comment has been minimized.

Show comment
Hide comment
@shruthimr

shruthimr Oct 28, 2013

@eluzix - I did the following still I don't see any audio being written out with the video file.

movieFile.playSound = YES;
movieWriter.shouldPassthroughAudio = NO;
movieFile.audioEncodingTarget = movieWriter;
[movieFile enableSynchronizedEncodingUsingMovieWriter:movieWriter];

I see you comment in GPUImageMovie.h .
/** This determines whether audio should be played. Can't be set to work with video writing. Defaults to NO.
*/
@Property(readwrite, nonatomic) BOOL playSound;

So does that mean I cannot add an audio track to my file ?

Thanks,
Shruthi.

@eluzix - I did the following still I don't see any audio being written out with the video file.

movieFile.playSound = YES;
movieWriter.shouldPassthroughAudio = NO;
movieFile.audioEncodingTarget = movieWriter;
[movieFile enableSynchronizedEncodingUsingMovieWriter:movieWriter];

I see you comment in GPUImageMovie.h .
/** This determines whether audio should be played. Can't be set to work with video writing. Defaults to NO.
*/
@Property(readwrite, nonatomic) BOOL playSound;

So does that mean I cannot add an audio track to my file ?

Thanks,
Shruthi.

@tuo

This comment has been minimized.

Show comment
Hide comment
@tuo

tuo Nov 1, 2013

@shruthimr make sure that this line

movieWriter.shouldPassthroughAudio = NO;

is set to YES rather than NO

If you set it to NO, there is not gonna have AVAssetWriterInput of AVMediaTypeAudio initialized, then of course there is no sound/audio be written to file

tuo commented Nov 1, 2013

@shruthimr make sure that this line

movieWriter.shouldPassthroughAudio = NO;

is set to YES rather than NO

If you set it to NO, there is not gonna have AVAssetWriterInput of AVMediaTypeAudio initialized, then of course there is no sound/audio be written to file

@shruthimr

This comment has been minimized.

Show comment
Hide comment
@shruthimr

shruthimr Nov 1, 2013

movieFile.playSound = YES;
movieWriter.shouldPassthroughAudio = YES;
movieFile.audioEncodingTarget = movieWriter;
[movieFile enableSynchronizedEncodingUsingMovieWriter:movieWriter]

Just plays the audio file music even before the actual file has been encoded. And the final file does not have any audio track.

movieFile.playSound = YES;
movieWriter.shouldPassthroughAudio = YES;
movieFile.audioEncodingTarget = movieWriter;
[movieFile enableSynchronizedEncodingUsingMovieWriter:movieWriter]

Just plays the audio file music even before the actual file has been encoded. And the final file does not have any audio track.

@eluzix

This comment has been minimized.

Show comment
Hide comment
@eluzix

eluzix Nov 3, 2013

hi @shruthimr as i mentioned in my original post only one of playSound and audioEncodingTarget will work. This behaviour is fine for our needs so we didn't try to extend it.

if you wish to play with it your self, take a look at GPUImageMovie readNextAudioSampleFromOutput: thats where the decision is made, you should change:

} else if (self.audioEncodingTarget != nil && !audioEncodingIsFinished){
    [self.audioEncodingTarget processAudioBuffer:audioSampleBufferRef];
}

to:

} 

if (self.audioEncodingTarget != nil && !audioEncodingIsFinished){
    [self.audioEncodingTarget processAudioBuffer:audioSampleBufferRef];
}

removing the else so both the audio player and the encoding target gets the buffer.

I haven't tried it myself and have no idea what might go wrong if you try but i suspect it should go fairly well.

eluzix commented Nov 3, 2013

hi @shruthimr as i mentioned in my original post only one of playSound and audioEncodingTarget will work. This behaviour is fine for our needs so we didn't try to extend it.

if you wish to play with it your self, take a look at GPUImageMovie readNextAudioSampleFromOutput: thats where the decision is made, you should change:

} else if (self.audioEncodingTarget != nil && !audioEncodingIsFinished){
    [self.audioEncodingTarget processAudioBuffer:audioSampleBufferRef];
}

to:

} 

if (self.audioEncodingTarget != nil && !audioEncodingIsFinished){
    [self.audioEncodingTarget processAudioBuffer:audioSampleBufferRef];
}

removing the else so both the audio player and the encoding target gets the buffer.

I haven't tried it myself and have no idea what might go wrong if you try but i suspect it should go fairly well.

@aiwo

This comment has been minimized.

Show comment
Hide comment
@aiwo

aiwo Nov 30, 2013

Hi @eluzix. I think I've faced the same problem. After processing a video file in movie writer there's no audio track at all. Your solution above making a crash with EXC_BAD_ACCESS because of invalidation of audioBuffer.
Trying to fix it right now.

aiwo commented Nov 30, 2013

Hi @eluzix. I think I've faced the same problem. After processing a video file in movie writer there's no audio track at all. Your solution above making a crash with EXC_BAD_ACCESS because of invalidation of audioBuffer.
Trying to fix it right now.

@nikhilcoolstuff

This comment has been minimized.

Show comment
Hide comment
@nikhilcoolstuff

nikhilcoolstuff Dec 3, 2013

Hi @shruthimr ..we're you able to add support for audio encoding target with @eluzix changes? I tried making the changes as @eluzix suggested in his last post but it doesn't work. Still I get no audio after encoding and the sound plays much before the encoding actually starts. Request any pointers...Thanks!

Hi @shruthimr ..we're you able to add support for audio encoding target with @eluzix changes? I tried making the changes as @eluzix suggested in his last post but it doesn't work. Still I get no audio after encoding and the sound plays much before the encoding actually starts. Request any pointers...Thanks!

@huize

This comment has been minimized.

Show comment
Hide comment
@huize

huize Sep 25, 2014

init:
currentGPUImageMovie=[[GPUImageMovie alloc] initWithPlayerItem:playerItem];
player=[[AVPlayer alloc] initWithPlayerItem:playerItem];
currentGPUImageMovieWriter=[[GPUImageMovieWriter alloc] initWithMovieURL:tempMoiveURL size:_myVideoView.bounds.size];

[currentGPUImageMovie enableSynchronizedEncodingUsingMovieWriter:currentGPUImageMovieWriter];
currentGPUImageMovie.audioEncodingTarget=currentGPUImageMovieWriter;
currentGPUImageMovieWriter.shouldPassthroughAudio=YES;
[currentGPUImageMovieWriter setHasAudioTrack:YES];

start write:
[player play];
[currentGPUImageMovieWriter startRecording];
[currentGPUImageMovie startProcessing];

why the final file not contain audio track ,i use the lastest version GPUImage ,please help

huize commented Sep 25, 2014

init:
currentGPUImageMovie=[[GPUImageMovie alloc] initWithPlayerItem:playerItem];
player=[[AVPlayer alloc] initWithPlayerItem:playerItem];
currentGPUImageMovieWriter=[[GPUImageMovieWriter alloc] initWithMovieURL:tempMoiveURL size:_myVideoView.bounds.size];

[currentGPUImageMovie enableSynchronizedEncodingUsingMovieWriter:currentGPUImageMovieWriter];
currentGPUImageMovie.audioEncodingTarget=currentGPUImageMovieWriter;
currentGPUImageMovieWriter.shouldPassthroughAudio=YES;
[currentGPUImageMovieWriter setHasAudioTrack:YES];

start write:
[player play];
[currentGPUImageMovieWriter startRecording];
[currentGPUImageMovie startProcessing];

why the final file not contain audio track ,i use the lastest version GPUImage ,please help

@jcampbell05

This comment has been minimized.

Show comment
Hide comment
@jcampbell05

jcampbell05 Oct 30, 2014

@eluzix I just added audioFormat.mSampleRate = asbd->mSampleRate; to the bit where you update the buffer format type and it plays correctly.

@eluzix I just added audioFormat.mSampleRate = asbd->mSampleRate; to the bit where you update the buffer format type and it plays correctly.

@eluzix

This comment has been minimized.

Show comment
Hide comment
@eluzix

eluzix Nov 6, 2014

@jcampbell05 thanks, as i no longer working with Pheed i can't really update the code. Why wont you fork it and make the update available?

eluzix commented Nov 6, 2014

@jcampbell05 thanks, as i no longer working with Pheed i can't really update the code. Why wont you fork it and make the update available?

@amayers

This comment has been minimized.

Show comment
Hide comment
@amayers

amayers Dec 12, 2014

To play audio all I had to do was:

NSURL *mediaURL = ...
AVPlayer *mainPlayer = [AVPlayer alloc] init];
AVPlayerItem playerItem = [[AVPlayerItem alloc] initWithURL:mediaUrl];
[mainPlayer replaceCurrentItemWithPlayerItem:playerItem];

GPUImageMovie *currentMovieFile = [[GPUImageMovie alloc] initWithPlayerItem:playerItem];

// start playback
[currentMovieFile startProcessing]
[mainPlayer play]

amayers commented Dec 12, 2014

To play audio all I had to do was:

NSURL *mediaURL = ...
AVPlayer *mainPlayer = [AVPlayer alloc] init];
AVPlayerItem playerItem = [[AVPlayerItem alloc] initWithURL:mediaUrl];
[mainPlayer replaceCurrentItemWithPlayerItem:playerItem];

GPUImageMovie *currentMovieFile = [[GPUImageMovie alloc] initWithPlayerItem:playerItem];

// start playback
[currentMovieFile startProcessing]
[mainPlayer play]

@jcampbell05

This comment has been minimized.

Show comment
Hide comment

@eluzix I have a fork available here.

https://github.com/Unii-Ltd/GPUImage

@danniss

This comment has been minimized.

Show comment
Hide comment
@danniss

danniss Feb 6, 2015

@jcampbell05
your fork breaks a lot of my existing code as it removes some fields from GPUImageMovie.h. what's the intention with it? may be something wrong with the code merge?

danniss commented Feb 6, 2015

@jcampbell05
your fork breaks a lot of my existing code as it removes some fields from GPUImageMovie.h. what's the intention with it? may be something wrong with the code merge?

@tspecht

This comment has been minimized.

Show comment
Hide comment
@tspecht

tspecht Apr 20, 2015

Any news on this issue? I need to make sure the audio and the video is synced and I'm not yet sure if GPUImage is the way for me to go

tspecht commented Apr 20, 2015

Any news on this issue? I need to make sure the audio and the video is synced and I'm not yet sure if GPUImage is the way for me to go

@UriChan

This comment has been minimized.

Show comment
Hide comment
@UriChan

UriChan Dec 28, 2015

@huize ,hi huize, i met the same problem, have you solved it yet?

UriChan commented Dec 28, 2015

@huize ,hi huize, i met the same problem, have you solved it yet?

@jcampbell05

This comment has been minimized.

Show comment
Hide comment
@jcampbell05

jcampbell05 Dec 29, 2015

@UriChan I made a fork above which worked :)

@UriChan I made a fork above which worked :)

@UriChan

This comment has been minimized.

Show comment
Hide comment
@UriChan

UriChan Jan 2, 2016

@jcampbell05, thanks, i check you fork, it works good for local file, but it does't work for a remote stream, "GPUImageMovie initWithPlayerItem" works with remote stream, but the GPUImageMovieWriter result file does't contain audio track. I have bothered by this problem for quite a long time, any idea how to solve it?

UriChan commented Jan 2, 2016

@jcampbell05, thanks, i check you fork, it works good for local file, but it does't work for a remote stream, "GPUImageMovie initWithPlayerItem" works with remote stream, but the GPUImageMovieWriter result file does't contain audio track. I have bothered by this problem for quite a long time, any idea how to solve it?

@jcampbell05

This comment has been minimized.

Show comment
Hide comment
@jcampbell05

jcampbell05 Jan 2, 2016

Not sure, in my case I downloaded the remote file and then played it. Didn't do it via a stream

Not sure, in my case I downloaded the remote file and then played it. Didn't do it via a stream

@Alfarache

This comment has been minimized.

Show comment
Hide comment
@Alfarache

Alfarache Feb 8, 2016

@jcampbell05 I am getting a 404 on your fork link above. Is it still available?

@jcampbell05 I am getting a 404 on your fork link above. Is it still available?

@jcampbell05

This comment has been minimized.

Show comment
Hide comment
@jcampbell05

jcampbell05 Feb 9, 2016

@Alfarache Ah I think my old company may have shut their repo down - I can ask if someone has a copy. @UriChan would you care to upload it if you still have it ?

@Alfarache Ah I think my old company may have shut their repo down - I can ask if someone has a copy. @UriChan would you care to upload it if you still have it ?

@geek-paulwong

This comment has been minimized.

Show comment
Hide comment
@geek-paulwong

geek-paulwong Jun 1, 2016

@jcampbell05 Do you still have a copy? Really appreciate if you could upload the working copy. Thank you!!

@jcampbell05 Do you still have a copy? Really appreciate if you could upload the working copy. Thank you!!

@jcampbell05

This comment has been minimized.

Show comment
Hide comment
@jcampbell05

jcampbell05 Jun 2, 2016

@geek-paulwong and @Alfarache I got hold of the source code, here you go:

GPUImage-master.zip

@geek-paulwong and @Alfarache I got hold of the source code, here you go:

GPUImage-master.zip

@geek-paulwong

This comment has been minimized.

Show comment
Hide comment

@jcampbell05 Thank you!!! ;)

@geek-paulwong

This comment has been minimized.

Show comment
Hide comment
@geek-paulwong

geek-paulwong Aug 25, 2016

@jcampbell05 Hey sir, good morning! For me, the source code actually crashed in readNextAudioSampleFromOutput: method, and hence did not really work. :(

Could you help me look into it? A million thanks!

@jcampbell05 Hey sir, good morning! For me, the source code actually crashed in readNextAudioSampleFromOutput: method, and hence did not really work. :(

Could you help me look into it? A million thanks!

@geek-paulwong

This comment has been minimized.

Show comment
Hide comment
@geek-paulwong

geek-paulwong Aug 25, 2016

@amayers Did it happen to you that GPUImageMovie occasionally goes blank, and nothing is showing up? It is happening to me from time to time. I have tried everything, but still could not fix it. Please help me if you know how to fix that. Thank you!

@amayers Did it happen to you that GPUImageMovie occasionally goes blank, and nothing is showing up? It is happening to me from time to time. I have tried everything, but still could not fix it. Please help me if you know how to fix that. Thank you!

@geek-paulwong

This comment has been minimized.

Show comment
Hide comment
@geek-paulwong

geek-paulwong Aug 26, 2016

@eluzix Thank you very much!! Your forked version of GPUImage that supports audio works perfectly for my needs!!

@eluzix Thank you very much!! Your forked version of GPUImage that supports audio works perfectly for my needs!!

@jiangqc5656

This comment has been minimized.

Show comment
Hide comment
@jiangqc5656

jiangqc5656 Dec 9, 2016

@geek-paulwong Hello, I met the same problem with you! Use GPUImageMovie play video, while using AVPlayer to play audio, there will be video and audio are not synchronized problem! Download the eluzix's code, but his code is too old, and now the GPUImage has been unable to compile. If you have solved this problem, you can modify the distribution of GPUImage?

@geek-paulwong Hello, I met the same problem with you! Use GPUImageMovie play video, while using AVPlayer to play audio, there will be video and audio are not synchronized problem! Download the eluzix's code, but his code is too old, and now the GPUImage has been unable to compile. If you have solved this problem, you can modify the distribution of GPUImage?

@geek-paulwong

This comment has been minimized.

Show comment
Hide comment
@geek-paulwong

geek-paulwong Dec 9, 2016

@jiangqc5656 Hey man, I used exactly the codes from @eluzix and it worked well for me. You are welcome to add me on WeChat. My username is paulwongcool. I'd be happy to give you more tips there to make it work.

@jiangqc5656 Hey man, I used exactly the codes from @eluzix and it worked well for me. You are welcome to add me on WeChat. My username is paulwongcool. I'd be happy to give you more tips there to make it work.

@CaichaoGitHub

This comment has been minimized.

Show comment
Hide comment
@CaichaoGitHub

CaichaoGitHub Nov 16, 2017

@pgodino hi, i have already play the audio using audio unit, i meet the same problem like you , i don't know how to sync the audio and video , do you have any solutions ?

@pgodino hi, i have already play the audio using audio unit, i meet the same problem like you , i don't know how to sync the audio and video , do you have any solutions ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment