Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streaming & Background threading on play #62

Merged
merged 11 commits into from
Jan 29, 2016
Merged

Streaming & Background threading on play #62

merged 11 commits into from
Jan 29, 2016

Conversation

ghenry22
Copy link
Contributor

This fixes 2 issues in the Cordova Jira, both with iOS. It enables streaming support for media playback from URLs while retaining the existing playback for local media.

The second part is the simply background thread the play function so it doesn't block the ui or lock up the app on load. Gets rid of some of those pesky warnings in xcode too.

Tested with simulator and on iphone 5 and iphone 6 and ipad 2, with iOS 8.1, 8.3 & 8.4

Please merge these and bump the version of the media plugin so I can stop installing them from my own fork :)

Fix for CB-57
Updated to use avplayer when url starts with http:// or https:// for
full streaming support.

All other urls will use the standard AVAudioplayer

Fix for CB-8222
Background thread on play to prevent locking during initial load of
media.

Fix for CB-57
Updated to use avplayer when url starts with http:// or https:// for
full streaming support.

All other urls will use the standard AVAudioplayer

Fix for CB-8222
Background thread on play to prevent locking during initial load of
media.
@johncblandii
Copy link
Contributor

I tested this on my app and the playback starts and pauses as expected [on a streaming m3u].

+1

@nigel-dev
Copy link

Been testing this and the stop() method does not seem to work, I had to use pause() to get the audio to actually stop when streaming an mp3 file.

@ghenry22
Copy link
Contributor Author

avplayer which is used for streaming does not have an equivalent stop method. You can just pause() and then release() there is no need to stop().

@nigel-dev
Copy link

Awesome thanks!

@purplecabbage
Copy link
Contributor

The API includes stop, if pause+release is the same thing, then your code should do that!
Otherwise you are breaking an api and forcing developers to change their app code.

@ghenry22
Copy link
Contributor Author

I'll look at putting something in that mimics the existing stop behavior applied to avaudioplayer.

Any chance of this actually getting merged once that is done?

@Alkashi
Copy link

Alkashi commented Sep 6, 2015

Hi,
Is there any news about the merge ?

NB : @ghenry22 the getDuration seems to not work

@leon
Copy link

leon commented Sep 29, 2015

+1
Please merge this :)

@ghenry22
Copy link
Contributor Author

ghenry22 commented Oct 8, 2015

Alkashi - getDuration definitely works as expected on ios devices. I use it every day in my app and have tested it extensively on multiple devices and on the simulator.

There is always the chance that some files won't work of course, but that is more a limitation of the underlying AVPlayer framework and not something we can do anything about.

@Alkashi
Copy link

Alkashi commented Oct 8, 2015

@ghenry22 OK Thanks for the feedback I'll try again asap.

@Alkashi
Copy link

Alkashi commented Oct 11, 2015

@ghenry22 I just tested and the getDuration() method which is supposed to return the total duration of the mp3 file return me always 0.
Tested on iPhone 6 - iOS9

On Android everything woks fine.

@ghenry22
Copy link
Contributor Author

This pull request doesn't make any changes to the behavior of get duration calls though, I see with my media some times 0 is returned for some files sometimes it's a full duration, seems to vary depending on what information is available in the file or stream.

@nigel-dev
Copy link

@Alkashi Im using this same code on iOS 9 and its working properly. Here is a code snippet on what I am doing.

function onPlayerStatus(status) {
    console.log("onPlayerStatus:" + status);
    if (status == 2) {
        $('#btnPlay').addClass("btn-play-active");
        currentDuration = player.getDuration();
        console.log("CURRENT DURATION: " + currentDuration);
        updateMetas(mediaButton.data('title'),section);
    } else if (status == 4) {
        clearInterval(mediaTimer);
        mediaTimer = null;
        $('#btnPlay').removeClass("btn-play-active");
        $('#circProgress').circleProgress('value', 0);
    }
};

@blakedietz
Copy link

+1 for @ghenry22 's fix. All you have to do is run a cordova plugin rm cordova-plugin-media and then cordova plugin add https://github.com/ghenry22/cordova-plugin-media.git#avplayer_streaming and streaming from a remote source such as soundcloud works.

@dts
Copy link

dts commented Oct 12, 2015

Good stuff - only downside is it currently always reports '0' as the duration of the resource. CDVSound should have if(avPlayer.currentItem && avPlayer.currentItem.asset) { CMTime time = avPlayer.currentItem.asset.duration; position = CMTimeGetSeconds(time); } instead of avPlayer.currentTime.

stop will now behave the same when avplayer is used to play from a
stream as when avaudioplayer is used to play from a local file.
seekto 0 then stop playing.
@ghenry22
Copy link
Contributor Author

@dts - thanks for the suggestion, I have implemented this and hopefully also fixed for playback from local files as well.

@purplecabbage - updated with implementation for stop() for compliance as requested.

incremented the version number so that I can see which version is actually installed. if you install from my branch it should show version 1.0.3-dev when you run cordova plugin list.

@Alkashi
Copy link

Alkashi commented Oct 13, 2015

OK so I was not completely crazy. Many thanks to @ghenry22 and others.

@ghenry22
Copy link
Contributor Author

@Alkashi - nope you were quite right. I realised I had a different branch of the plugin installed myself to test some things for Android and it was giving me correct durations, Hence I bumped the version number so I could see what I have running!

@nigel-dev
Copy link

@Alkashi @ghenry22 I did the same thing, at some point I patched the version that I'm using in my project. Oops :)

@ghenry22
Copy link
Contributor Author

ghenry22 commented Nov 5, 2015

PR #12 & PR #18 could also be closed out as duplicate once this one is merged which will help to clean up some older lingering PR's

@ghenry22
Copy link
Contributor Author

Not sure about soundcloud as I haven't tried to stream from them before. Whether you can seek in a stream or not would probably depend on the source and the meta data it provides and if that is enough for the avplayer underlying framework of Apples to be happy.

One case I have come across on Android with some streams is that if the length of the stream is unknown it assigns it a length of 0 and then any attempt to seek or a stall in streaming and then resume will reset back to start, I haven't hit the same with iOS yet on those same streams but there are probably other quirks / scenarios where the iOS framework would have a similar issue.

@ghenry22
Copy link
Contributor Author

seekTo definitely works on streams from other sources, my primary test is the subsonic media server but I have tried other random sources like free sample streams from spotify which also worked for me.

@dduwab
Copy link

dduwab commented Nov 21, 2015

I'm having trouble hearing sound from a local mp3 on iOS. I can play a sound from a http:// address but not from the local file tree www/assets/titleMusic.mp3
Resetting plugins due to page load. Log:
Finished load of: file:///Users/.../Devices/.../data/Containers/Bundle/Application/.../ccgame.app/www/main.html
Found resource '/Users/.../Devices/.../data/Containers/Bundle/Application/.../ccgame.app/www/assets/titleMusic.mp3' in the web folder.
Playing audio sample 'assets/titleMusic.mp3'
Playing stream with AVPlayer

Yet I don't hear any sound in this case. I'm new to iOS development but I did my best to trace through the code and no errors appeared and everything looked like it should be playing.
Again, there were no problems playing/hearing a file off the web.
Thanks.

@ghenry22
Copy link
Contributor Author

It's trying to play a local file as a stream due to the path being used by the look of it. Please try using the cordova.file.* paths to get a cdvfile:// path to your audio file and play with that path, should resolve your issue and make sure that your app works properly across platforms and installs as the path is consistent.

Check the file plugin for details on the file paths.

Cheers
Gaven

On 21 Nov 2015, at 8:29 PM, dduwab notifications@github.com wrote:

I'm having trouble hearing sound from a local mp3 on iOS. I can play a sound from a http:// address but not from the local file tree www/assets/titleMusic.mp3
Resetting plugins due to page load. Log:
Finished load of: file:///Users/.../Devices/.../data/Containers/Bundle/Application/.../ccgame.app/www/main.html
Found resource '/Users/.../Devices/.../data/Containers/Bundle/Application/.../ccgame.app/www/assets/titleMusic.mp3' in the web folder.
Playing audio sample 'assets/titleMusic.mp3'
Playing stream with AVPlayer

Yet I don't hear any sound in this case. I'm new to iOS development but I did my best to trace through the code and no errors appeared and everything looked like it should be playing.
Again, there were no problems playing/hearing a file off the web.
Thanks.


Reply to this email directly or view it on GitHub.

@NurdinDev
Copy link

How Allow IOS Support Streaming?

@ghenry22
Copy link
Contributor Author

ghenry22 commented Jan 4, 2016

@nuruddinbadawi install my branch of the media plugin with this command:

cordova plugin add https://github.com/ghenry22/cordova-plugin-media.git#avplayer_streaming

That will get you the version with streaming support

@NurdinDev
Copy link

Skipping 'cordova-plugin-media' for android
"Plugin doesn't support this project's cordova-android version.
cordova-android: 4.1.1, failed version requirement: >=5.0.0-dev "

On Mon, Jan 4, 2016 at 4:14 AM, Gaven Henry notifications@github.com
wrote:

@nuruddinbadawi https://github.com/nuruddinbadawi install my branch of
the media plugin with this command:

cordova plugin add
https://github.com/ghenry22/cordova-plugin-media.git#avplayer_streaming

That will get you the version with streaming support


Reply to this email directly or view it on GitHub
#62 (comment)
.

@ghenry22
Copy link
Contributor Author

ghenry22 commented Jan 4, 2016

update your android platform and try again

@NurdinDev
Copy link

in ios work fine but in android no

can install this brush just in ios and let android use default Media Plugin
because I don't have any problem with streaming in android

and thanks

On Mon, Jan 4, 2016 at 9:28 AM, Gaven Henry notifications@github.com
wrote:

update your android platform and try again


Reply to this email directly or view it on GitHub
#62 (comment)
.

This reverts commit e7f9f04.
This will still intermittently fail test 19 but this is more to do with
the success criteria of that rest.  rate is applied and works.
@ghenry22
Copy link
Contributor Author

@purplecabbage I have added rate control when streaming from a URL using avplayer and tested that it works. However test case 19 still fails in the automated suite but I believe that is due to the way media is loaded and the success evaluator on the test.

To explain:
When setting rate to 0.5 and listening to a file it is audibly slower than normal playback rate.
When setting rate to 2.0 and listening to a file it is audibly faster than normal playback rate.
So rate can be set and adjusted and does change playback rate.

The problem with the test is it plays for 4 seconds and expects the playback position to be >7 when playing at rate 2. This is fine when using the old avaudioplayer implementation as it will first download the entire file locally and then start paying immediately at rate 2, resulting in a final position generally around 8.

When using avplayer though to stream properly we don't download the entire file in advance, we only start streaming when play is pushed so we now have a variance that throws the test out. If the file is slow to load from the source server for some reason we have a larger variance. GIven only 4 seconds play time there is not enough time for the stream to start buffering and play at 2x rate in order to reach the magic test passing number. This is completely normal and expected.

If the length of the test run was extended to say 10 seconds you would find a lot less variance as it allows some time to buffer.

If the mp3 source file is hosted on a local http server on the same machine running the emulator and tests it will also download almost instantly and pass the test. (I have tested this scenario).

To test rate you could potentially play for 4 seconds at rate 1 and then play for 4 seconds at rate 2 and if the output of the second play is close to 2x the first play then you have success. Given that the tests are run one after the other they are likely to both have the same buffering time which mostly rules that out as a variable.

I am not sure how to rewrite the test to do this though. If someone could help to rewrite the test then all tests will pass and this PR can finally get merged which would be fantastic!

@imsingh
Copy link

imsingh commented Jan 28, 2016

@ghenry22 Hey, I am not able to use the .m3u stream on iOS. I am getting following error.

2016-01-28 23:44:41.939 MPESG Radio[3073:44952] Failed to initialize AVAudioPlayer: The operation couldn’t be completed. (OSStatus error 1954115647.)
2016-01-28 23:44:41.940 MPESG Radio[3073:44952] THREAD WARNING: ['Media'] took '3625.225098' ms. Plugin should use a background thread.

@ghenry22
Copy link
Contributor Author

@imsingh That doesn't really have anything to do with this pull request. I'm not sure what "MPESG Radio" is that is generating the error messages, never seen that one before. From my experience with the media plugin you need to give it a single file or single source URL to play, I don't think it will handle an m3u file with multiple items. No idea what it would do with an m3u with multiple entries.

You could always grab the m3u file, strip out the list of items and then manage them as a playlist in your app code feeding them to the media plugin as sources appropriately.

Oh also it looks like you are either loading the m3u file locally or using the main branch of the media plugin, not my branch with the streaming code as if the resource is loaded from a HTTP:// url with this branch then it will use avplayer not avaudioplayer.

@imsingh
Copy link

imsingh commented Jan 29, 2016

@ghenry22 MPESG Radio is just name of Cordova App and m3u file has a single URL entry. I Tried using the URL Directly again it is not working. However the plugin at https://github.com/keosuofficial/cordova-audio-stream-plugin works fine, but it has not implemented callbacks so I can't use it either.

@ghenry22
Copy link
Contributor Author

@imsingh where are you loading the m3u file from? If you are opening the m3u file from the local file system it will try to play it like a local file. If you open it from a http url then it will try to stream it. This is just the logic in the media plugin, it looks at the source prefix to determine if it should stream or play locally.

If you haven't already done so I have a fork of the plugin here that you can install using the command below, this fork has the updated ios streaming code from this pull request build in. If you install from here and you're m3u file is being loaded from a url that starts with http or https you should see a message in the native console saying "playing stream with avplayer" instead of the normal message you see which is tied to "avaudioplayer". avaudioplayer is the original implementation that does not support streaming, avplayer is the framework implemented in this PR to add proper streaming support.

cordova plugin add https://github.com/ghenry22/cordova-plugin-media.git#avplayer_streaming

that's the best that I can think to recommend, hope it helps a bit!

@ghenry22
Copy link
Contributor Author

@purplecabbage sorry to bug you but if you could have a look at my previous comment that would be great, I really want to get this merged as it has been hanging for a long time and is starting to be difficult to maintain my fork and keep everything up to date.

@asfgit asfgit merged commit 90f4623 into apache:master Jan 29, 2016
@purplecabbage
Copy link
Contributor

Good to go. Thanks for sharing your work @ghenry22 !

@johncblandii
Copy link
Contributor

Sweet!!

@ghenry22
Copy link
Contributor Author

ghenry22 commented Feb 1, 2016

Awesome! Glad I could contribute something back to the project!

@Taewa
Copy link

Taewa commented Feb 29, 2016

My streaming player got much faster. Thank you.
But I had to install this version through
sudo cordova plugin add cordova-plugin-media@2.2

I originally tried with
sudo cordova plugin add cordova-plugin-media

But didn't work.

@purplecabbage
Copy link
Contributor

We are further and further off topic, but
Why are you using sudo to install plugins @Taewa ?

@ghenry22
Copy link
Contributor Author

ghenry22 commented Apr 8, 2016

@purplecabbage comments on this PR could not possibly go further off topic :) Seems to have become defacto media plugin support for a while there.

I had some odd permission issue on my mac initially and used sudo to install as well. No matter what I did always had that problem, probably because I installed something early on with Sudo and then all the permissions became a mess.

Was resolved when I did a refresh of the OS on my mac and a clean install of all the tools without using sudo for anything.

Doesn't seem to hurt anything either way though and all build tools etc work.

@Taewa
Copy link

Taewa commented Apr 14, 2016

Hello @purplecabbage
I actually didn't know why I had to use sudo.
As @ghenry22 explained, I had odd permission problem. Some project had to use sudo, some are not.
I tried chmod but, couldn't resolve this issue

@purplecabbage
Copy link
Contributor

Thanks @Taewa ! I was just curious. I wonder how often this happens?! Never on windows ... ;)

@ChromaticMind
Copy link

hi @ghenry22, must say your work here is stellar!

I have 2 issues with this plugin so far (i am using the latest pull from the main repo as your work is now merged there)

  1. on iOS a streamling mp3 (30min long podcast) it can take upto 10secs for the track to start playing. On android it's no more than 2 seconds.. if i put the same link on a browser it will start playing in less than a second

  2. if i only play local files or only streaming urls then its fine. if i stream something and then play a local file it will start playing the stream again - again only on ios. Android works fine so i know it's a plugin thing and not something in the code as it's the same code for both ( i do remove the "file://" for ios local tracks)

Any help here would be immensely appreciated @ghenry22 . Many many thanks!

@nudou350
Copy link

nudou350 commented Jul 9, 2018

Hey! I can't use the "cordova plugin add https://github.com/ghenry22/cordova-plugin-media.git#avplayer_streaming" command, it says:
npm ERR! Command failed: C:\Program Files\Git\cmd\git.EXE checkout avplayer_streaming
npm ERR! error: pathspec 'avplayer_streaming' did not match any file(s) known to git.

Am I missing something?

@purplecabbage
Copy link
Contributor

That branch ( and this pr ) has been merged. @ghenry22 's branch no longer exists.
Just install via cordova plugin add cordova-plugin-media
https://github.com/ghenry22/cordova-plugin-media/branches

@nudou350
Copy link

nudou350 commented Jul 9, 2018

Thanks for the info! But adding that works in browser and emulated android device. But when i try on a real device, it takes around 20s to start playing the stream! Any tips?

@onneti2013
Copy link

onneti2013 commented Aug 13, 2019

Hi use cordova plugin add cordova-plugin-media@2.2 solved slowness start

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.