Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Android suggestion for audio player #3

Closed
rohansohonee opened this issue Dec 4, 2019 · 30 comments
Closed

Android suggestion for audio player #3

rohansohonee opened this issue Dec 4, 2019 · 30 comments
Assignees

Comments

@rohansohonee
Copy link

Hi @ryanheise

Can you use exoplayer at android side instead of media player.
Exoplayer is very feature rich.

@ryanheise
Copy link
Owner

@rohansohonee , I was planning to implement much the same using the low level MediaExtractor and AudioTrack APIs which would give me more flexibility in implementing the features I want.

E.g. this way, I can implement time stretching and also the untilPosition parameter of play which isn't built in to exoplayer. However, I would consider exoplayer if it were easy to extend it in this way. Do you know anything of its extensibility framework and how one would go about plugging this feature in?

@ryanheise ryanheise self-assigned this Dec 4, 2019
@rohansohonee
Copy link
Author

I am not clear about the untilPosition parameter of play ?
Can you explain what untilPosition does?

You can customize exoplayer

@ryanheise
Copy link
Owner

@ryanheise
Copy link
Owner

Hmm, I didn't actually document it well, there, either! Basically, it will play until the given position, then pause.

@ryanheise
Copy link
Owner

By the way, are there specific features that you would like exposed in the flutter API?

@rohansohonee
Copy link
Author

If you can expose Exoplayer features in flutter API that would be awesome.

IMPORTANT
Please see this ryanheise/audio_service#114

@ryanheise
Copy link
Owner

@rohansohonee I am not going to add all of exoplayer's features in just_audio, as it comes with a lot of baggage that is not needed. Can you list specific features you would like to use?

@rohansohonee
Copy link
Author

Gapless playback and Normalize volume if possible.

Also have you seen the memory leak report?
ryanheise/audio_service#114

@ryanheise
Copy link
Owner

If it's just those two features (in addition to the two already mentioned up the top), I may just implement them myself without bringing in exoplayer, but I'll keep the exoplayer option in mind.

(Yes, I have seen your issue on audio_service.)

@ryanheise
Copy link
Owner

ryanheise commented Dec 25, 2019

I've just implemented time stretching and untilPosition on the iOS side (edit: sorry, I meant "Android" side). Gapless playback is possible by creating multiple instances and preparing the next instance for playback in advance.

I haven't looked at normalising the volume. How exactly should this behave?

@rohansohonee
Copy link
Author

Merry Christmas @ryanheise

I haven't looked at normalising the volume. How exactly should this behave?

  • Normalizing volume should keep a consistent volume between tracks. If it is possible to implement this then do so.

Gapless playback is possible by creating multiple instances and preparing the next instance for playback in advance.

  • I do this using the audioplayers plugin. The issue is it glitches the sound when playing the prepared instance. The reason for the glitch is that it uses MediaPlayer on android. Are you using MediaPlayer or AudioTrack??

I've just implemented time stretching and untilPosition on the iOS side.

  • Is this available for android as well?

@ryanheise
Copy link
Owner

I'm using AudioTrack. Let me know how it goes with gapless playback. If it still glitches, I'll look at building this feature into the plugin by using a single, continuous AudioTrack between files.

I should correct my earlier comment: I implemented untilPosition and time stretching on the "Android" side. I haven't yet done it on the iOS side.

Thanks for clarifying the normalised volume. I think it's a good idea, but it would require some time consuming and CPU intensive lookahead to determine the average energy of an audio file. If you have any references to articles etc. on how it could be done, that would be welcome.

@rohansohonee
Copy link
Author

rohansohonee commented Dec 25, 2019

I just know that exoplayer has the feature for normalising volume.

also rename untilPosition to just position.

@ryanheise
Copy link
Owner

Thanks, I'll look at what exoplayer does.

I don't like position as a parameter name for this method because it could easily be confused for a seek position, given that both the seek and until positions could conceivably be added to this method in the future.

An alternative could be to remove this parameter and have a separate method to set this position. Then the parameter itself could be named position.

@rohansohonee
Copy link
Author

  • It seems we can never start playback of audio as you have added a condition as follows:

Plays the currently loaded media from the current position. It is legal to invoke this method only from one of the following states:
[AudioPlaybackState.stopped]
[AudioPlaybackState.paused]

  • This means we can never start the initial playback. We never are in either stopped or paused.

@ryanheise
Copy link
Owner

After setUrl you should be in the stopped state.

@ryanheise
Copy link
Owner

One more thing which is not yet documented is that the future returned by play will complete when playback completes.

@rohansohonee
Copy link
Author

rohansohonee commented Dec 25, 2019

I created two instances of AudioPlayer().
If i stop player A it also stops B??
Multiple audio players should be possible to run.

@rohansohonee
Copy link
Author

Also the glitch occurs when calling play of player B while player A is running.

@rohansohonee
Copy link
Author

There is no AudioPlaybackState.completed state.
Also the onplayercompletion stream listener should be added.
Gapless playback not working since glitch is present when swapping previous song with next song.

@rohansohonee
Copy link
Author

Reason for gapless audio playback is because i want to cross fade between two tracks i.e fade out volume of current song while simultaneously fading in the next song, thus performing smooth transition.

@rohansohonee
Copy link
Author

Hi @ryanheise

I went through the android code and these are the changes you can make:

  1. Don't use MediaExtractor for decoding audio, but instead use FFmpeg for decoding the audio file.
    here is an article explaining the huge performance improvement.
    https://medium.com/@donturner/using-ffmpeg-for-faster-audio-decoding-967894e94e71
  2. Also you should really look into just using ExoPlayer as you will not have to make all these things from scratch.
  3. FFmpeg is also cross platform which is very useful for flutter.

@ryanheise
Copy link
Owner

Hi @rohansohonee

I did look into using ExoPlayer, but I did not see a way to implement untilPosition (which I need) without basically rewriting that whole part of the implementation anyway, and even then I would not be able to plug this into their interface because they designed their API without the forethought of having a feature like this. In the end, I am familiar enough with the low level Android APIs that it was going to be easier to just use those APIs. I understand that ExoPlayer is not so much of a "feature request" as it is an "implementation suggestion", and I'll still keep it on the table in the future, but for now it was much quicker for me to implement untilPosition this way. For now, I will just focus on features.

FFmpeg sounds good in theory but its license may get in the way of most developers of commercial apps actually being able to use it. See for example: https://trac.ffmpeg.org/ticket/1229 . If it weren't such an issue, I would definitely consider it. After all, FFmpeg would also result in lower battery usage. But in my opinion any cost of battery usage will be minor compared to battery used by the screen, and general performance is not going to be noticeable for the use case of just playing an audio file. I think the performance of FFmpeg would be useful when decoding a whole file in one go for analysis, e.g. to compute the average energy in order to normalise volume, but given the licensing issue, I would prefer to find a cheaper approach that can be done on the fly.

Speaking of normalisation, I wasn't able to find this feature in ExoPlayer. I found an issue on it, but it was closed without resolution: google/ExoPlayer#4541 . If you have any information on where it is implemented, I can take a look.

@rohansohonee
Copy link
Author

How is exoplayer using ffmpeg extensions then???

How do i perform gapless playback and Crossfade using your package?? I have tried two instances, where i am starting player B and stopping & immediately starting player A and then stopping player B to try gapless playback.
But there is always a glitch in audio. The transition is never smooth.

@rohansohonee
Copy link
Author

https://github.com/google/ExoPlayer/tree/release-v2/extensions/ffmpeg

The source code of ffmpeg extensions used in exoplayer.
Every android music app on play store uses exoplayer. They don't seem to be running into license issue??

@ryanheise
Copy link
Owner

It is mentioned on that page:

License note

Please note that whilst the code in this repository is licensed under
[Apache 2.0][], using this extension also requires building and including one or
more external libraries as described below. These are licensed separately.

FFmpeg itself is licensed under the LGPL.

However, this is just an optional extension and not everyone using ExoPlayer will be using FFmpeg.

@rohansohonee
Copy link
Author

So how do apps that are using it achieve it??

@ryanheise
Copy link
Owner

The general consensus online is that you should seek legal advice :-)

@rohansohonee
Copy link
Author

can you help with this issue if possible:

How do i perform gapless playback and Crossfade using your package?? I have tried two instances, where i am starting player B and stopping & immediately starting player A and then stopping player B to try gapless playback.
But there is always a glitch in audio. The transition is never smooth.

@github-actions
Copy link

github-actions bot commented Nov 8, 2021

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs, or use StackOverflow if you need help with just_audio.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants