fix: [iOS] seek pauses playback

[jramer]

Fix for #518

• Removes the pausing on seek which improves seek behaviour. Works for downloaded and streaming books but still sometimes pauses when streaming. (seems related to not getting data fast enough and errors from db)
• Tries to improve on sending metadata by using the same data structure as android.
• Removes some warnings

Needs some extra testing. Works with my .m4b and .mp3 books but would be good to try with many files (switching files while seeking) and maybe older ios versions.

[jramer]

BTW would be great with some funky audio books for testing. With many files, different formats and unusual metadata.

[jramer]

I have an idea on how to fix the pausing for streaming books that I want to try before merging this.

[jramer]

Tried to implement showing in the UI when player was buffering but can't find a reliable way of getting the metadata relay to js to update properly. Will keep tinkering but make a new PR if I get it to work.

[advplyr]

Are you sure this is required? If I remember correctly this flag was causing issues with playing HLS transcodes.
I could be wrong on that so I'm wondering what caused you to enable this.

[jramer]

I'm new to AVPlayer and streaming audio. Trying to read up on how it all works.
I'll try to make sure to test with HLS streams and it would be great if you can do that too.

But when I read the docs I came away wondering why you decided to set it to false. Could the HLS issues be connected to other decisions on how to handle the playing?

The docs say:

When playing media delivered over HTTP, this property is used to determine 
if the player should automatically delay playback in order to minimize stalling.

So when set to true the player tries to handle stalls itself.
If we set it to false we should handle it ourselves:

You will need to set this property to false when you require precise control over playback start times, 
such as if you're are synchronizing multiple player instances using the setRate(_:time:atHostTime:) method.

And we're not handling it.

It's good to note that:

If the value of this property is false, playback will start immediately when requested 
as long as the playback buffer is not empty.

So set to false the playback will start faster but, with the current code, we could end up needing to buffer and the player will be paused. And this is why my code had issues while streaming.

If you try my PR and compare it to the old code while seeking you will feel how much better the player behaves.

[advplyr]

I will do some testing today. HLS may be a problem because we are starting playback before the full audiobook is transcoded. I never figured out exactly how AVPlayer is handling this because their documentation is so slim, but what happens is it will request segments that haven't been created on the server yet.

On Android we can control how far out it will buffer and same with the website. AVPlayer tries to load way too much and when the server responds with an error that it hasn't been created yet, then it stalls the playback.

It has been a while now since I dove into this so I'll look into it again today now that I have a little more experience with iOS.

[advplyr]

I've been messing with this for about an hour testing the current release and this PR. I have only been testing direct streaming an audiobook with many mp3 files.

What I'm finding is that jumping forward/backward is much smoother on this PR as long as it is on the same audio track.

On this PR:
If you are 3 seconds into an audio track then jump back 10 seconds it will jump to the start of the current audio track. That is when seeking while paused. If you are playing and jump back 10 seconds to the previous track then it will usually jump to the start of the previous audio track.

On the current release:
When jumping backward to a previous track it is all over the place and I can't get consistent behavior. Sometimes it works correctly, sometimes it doesn't respond at all, and sometimes it restarts the current track.

I tried digging into this a bit and I could see at least some of it on the current release has to do with Realm. I wasn't able to get something working.

[jramer]

Ok interesting.
I’ll try to look at it.

[jramer]

Should be fixed now.
Seeking over item boundaries only changed item. Didn't seek in the new item...
Also fixed it to not loop around to first item when seeking past end of book.

[jramer]

I need help creating a book that streams as HLS.
I've tried some variations of this:
ffmpeg -y -i book.mp3 -c:a aac -b:a 128k -f hls -hls_time 120 -hls_list_size 0 -hls_segment_filename file%d.m4a playlist.m3u8

But playback fails (onPlaybackFailed) before it's loaded in the app.

[advplyr]

Thanks, I fixed an issue that was preventing the fallback to transcoding. However, there is another issue with the HLS stream that I confirmed is an issue regardless of the automaticallyWaitsToMinimizeStalling.

I think that the HLS stream issue is a general problem with the native iOS audio player. We can't control how far out the audio player is requesting segments, so if you watch the server it is going to continually reset the ffmpeg transcode because the requested segment is far from being created. I think we can make an update on the server to help with this but unless we can get some more fine grain control over the iOS player this will be a problem. Either way, not something to solve in this PR.

There was one other issue that I noticed and is also an issue on the current release.

While playing and you are ~5 seconds into an audio track jump back 10s to the previous track. This looks to jump back correctly and play at the end of the previous track, but once it goes to the next track it will not start at the beginning. It behaves as if the time on that audio track before you jumped back is persisted and once you move onto the track it starts back up on ~5s.

Overall this seems much smoother then previously and now functions better when navigating between tracks. Unless you know of a quick fix to that issue this is ready to merge since I'd like to release today. Thanks for your work on this!

[advplyr]

Oh btw if you want to test transcoding you can download a sample OPUS or OGG audio file which won't be playable in the native iOS app. That will cause the playback to fail and trigger the fallback to requesting a transcode from the server.

[advplyr]

Also to clarify, the problem with transcoding is we are transcoding on the fly. That means we "trick" the audio player to think the entire transcode is finished before it is. I wrote a little blurb about it a while ago here https://github.com/advplyr/hls-media-server#what-is-the-problem

[jramer]

Thanks! Nice to be able to help. Really like audiobookshelf.
Great with the sample files. I'll try to look at it and also that strange playback bug after seeking.