Support for parsing m4a chapters

[40ZqTYY4ag3tAO4hdlxR1]

Checklist

• [x ] I have used the search function to see if someone else has already submitted the same bug report.
• [x ] I will describe the problem with as much detail as possible.
• [x ] If the bug only to occurs with a certain podcast, I will include the URL of that podcast.

System info

[ Environment ]
Android version: 9
OS version: 3.10.61-TeamNexus-flx-00210-g4cebbfb8cc42-dirty
AntennaPod version: 1.8.1
Model: SM-G920F
Device: zeroflte
Product: zerofltexx
No crash report recorded

App version

AntennaPod 1.8.1 (6e6a64f) ( tried versions from F-Droid and GooglePlayStore )

Android version:
9 ( Lineage OS 16.0 )

Device model:
SM-G920F

Bug description

App is unable to detect chapters/bookmarks in some podcast files (M4A) when chapter data is encoded at the end of audio file.
The app looks for chapter data in the beginning of the file, therefore is unable to detect chapters if "chapter data" is encoded at the end of the M4A. I have noted this issue with a few "Apple Podcast M4A" files, new apple non standard file shenanigans?

Steps to reproduce:
Replicate using this file http://audiocdn.economist.com/sites/default/files/AudioArchive/2020/20200725/Issue_9204_20200725_The_Economist_03_Leaders.m4a
run ffprobe to confirm that the file does contain chapter markers ffprobe -i Issue_9204_20200725_The_Economist_03_Leaders.m4a -show_chapters -loglevel error

This is how the Podcastaddict dev described the issue and fixed it, (devs feedback follows).[...] The issue is that this specific podcast decided to move the chapter information at the end of the file instead of having them at the beginning which doesn't make any sense as it means that the app needs to get the whole file in order to just extract the metadata information. This podcast being especially long this would take a very long time and use a lot of data when streaming. I updated the app to be able to get those informations after the file gets downloaded as a workaround. [...]

Expected behaviour:

Chapters/bookmarks in audio file should be detected and should be displayed

Current behaviour:

Is unable to detect chapters/bookmarks in some podcast files when
First occurred:
New Install. Since install ( user migrated over from PodcastAddict ).
Environment:
New install, default settings used.
Stacktrace/Logcat:

Not applicable, app did not crash

[ByteHamster]

Relevant lines from the log:

07-25 08:42:50.434 11014 11068 D ChapterUtils: Reading id3 chapters from item ...
07-25 08:42:50.437 11014 11068 D ID3ChapterReader: No tag header found
07-25 08:42:50.437 11014 11068 I ChapterUtils: ChapterReader could not find any ID3 chapters
07-25 08:42:50.437 11014 11068 I ChapterUtils: Chapters loaded
07-25 08:42:50.438 11014 11068 D ChapterUtils: Trying to read chapters from item with title ...
07-25 08:42:50.440 11014 11068 I System.out: No vorbis comment found
07-25 08:42:50.440 11014 11068 I ChapterUtils: ChapterReader could not find any Ogg vorbis chapters

[40ZqTYY4ag3tAO4hdlxR1]

The file has chapters, but ID3ChapterReader does not see any. Not sure why :(

`ffprobe -show_chapters http://audiocdn.economist.com/sites/default/files/AudioArchive/2020/20200725/Issue_9204_20200725_The_Economist_03_Leaders.m4a

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x555be7552ec0] Unknown cover type: 0x0.

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'http://audiocdn.economist.com/sites/default/files/AudioArchive/2020/20200725/Issue_9204_20200725_The_Economist_03_Leaders.m4a':

Metadata:
major_brand : mp42
minor_version : 0
compatible_brands: mp42isom
creation_time : 2020-07-23T17:27:49.000000Z
title : 03 Leaders
artist : The Economist
album : July 25th-31st 2020
genre : Speech
track : 3/21
Duration: 00:32:30.16, start: 0.000000, bitrate: 64 kb/s
Chapter #0:0: start 0.000000, end 30.537000
Metadata:
title : 004 Leaders
Chapter #0:1: start 30.537000, end 530.612000
Metadata:
title : 005 Leaders - Macroeconomics
Chapter #0:2: start 530.612000, end 848.703000
Metadata:
title : 006 Leaders - Collective debt
Chapter #0:3: start 848.703000, end 1265.657000
Metadata:
title : 007 Leaders - Regional development
Chapter #0:4: start 1265.657000, end 1598.170000
Metadata:
title : 008 Leaders - Tanzania
Chapter #0:5: start 1598.170000, end 1950.162000
Metadata:
title : 009 Leaders - Space exploration
Stream #0:0(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 48 kb/s (default)
Metadata:
creation_time : 2020-07-23T17:27:49.000000Z
Stream #0:1(und): Data: bin_data (text / 0x74786574)
Metadata:
creation_time : 2020-07-23T17:27:49.000000Z
Unsupported codec with id 100359 for input stream 1
[CHAPTER]
id=0
time_base=1/1000
start=0
start_time=0.000000
end=30537
end_time=30.537000
TAG:title=004 Leaders
[/CHAPTER]
[CHAPTER]
id=1
time_base=1/1000
start=30537
start_time=30.537000
end=530612
end_time=530.612000
TAG:title=005 Leaders - Macroeconomics
[/CHAPTER]
[CHAPTER]
id=2
time_base=1/1000
start=530612
start_time=530.612000
end=848703
end_time=848.703000
TAG:title=006 Leaders - Collective debt
[/CHAPTER]
[CHAPTER]
id=3
time_base=1/1000
start=848703
start_time=848.703000
end=1265657
end_time=1265.657000
TAG:title=007 Leaders - Regional development
[/CHAPTER]
[CHAPTER]
id=4
time_base=1/1000
start=1265657
start_time=1265.657000
end=1598170
end_time=1598.170000
TAG:title=008 Leaders - Tanzania
[/CHAPTER]
[CHAPTER]
id=5
time_base=1/1000
start=1598170
start_time=1598.170000
end=1950162
end_time=1950.162000
TAG:title=009 Leaders - Space exploration
[/CHAPTER]
`

[ByteHamster]

AntennaPod currently does not support m4a chapters at all. If a podcast specifies the chapters in the RSS feed, they are shown independently from the media file, though. We already have m4a test files, so the only thing missing is writing the parser (see this m4a specification)

[antennapod-bot]

This issue has been mentioned on AntennaPod Forum. There might be relevant details there:

https://forum.antennapod.org/t/ldn-no-pictures-for-chaptermarks-in-the-m4a-feed-ldn311/2489/3

[antennapod-bot]

This issue has been mentioned on AntennaPod Forum. There might be relevant details there:

https://forum.antennapod.org/t/show-notes-for-local-folder-podcasts/2589/5

[HFPasfho]

I'm so down for that. I listen to circa 20 podcasts and none of them provides chapters supported by AnthennaPod. Actually I found out about AnthennaPod supporting chapters from this issue tracker because my podcasts only have m4a chapters if any and only some of them provide timestamps via RSS.

I want to add to this issue that chapter information can also be embedded into MP3 files. Maybe it's a good idea to extend this proposal by adding support for MP3 chapters alongside M4A.

[ByteHamster]

Chapters in MP3, MP4, Ogg and Opus are already supported.

[HFPasfho]

@ByteHamster Well hard luck because they're not for me apparently. I can confirm that the whole episode is treated as one big chapter despite that ffprobe reads chapters with no problem.

[HFPasfho]

Of course I have the latest version of AnthennaPod (2.7.1)

[ByteHamster]

This usually means that the podcast creator messed something up with the chapters (AntennaPod uses fewer workarounds than ffprobe), but feel free to create a new issue describing the problem in more detail.

[dstapp]

Can someone please clarify what the state of embedded M4A chapters is? This issue has two different positions:

28 Feb 21: "AntennaPod currently does not support m4a chapters at all."
Today: "Chapters in MP3, MP4, Ogg and Opus are already supported."

Was support added in the meantime (just asking because the issue is still open). I have the same problem that for some of my podcasts, chapters are not visible, although I can see them in Pocket Casts, Apple Podcasts and also with ffprobe when checking the m4a file. Would be awesome to get that working.

If I can provide any information to help figure this out, please let me know.

[40ZqTYY4ag3tAO4hdlxR1]

28 Feb 21: "AntennaPod currently does not support m4a chapters at all." Today: "Chapters in MP3, MP4, Ogg and Opus are already supported."

Credit to @ByteHamster , AntenanPod does support m4a chapters now. Have seen chapters in multiple podcasts now. Here is an example 'https://www.bloomberg.com/news/audio/2023-04-06/imf-western-alliance-etfs-and-kurodanomics-podcast' , the chapters are recognized, they show up as a list and in the little blue timeline bar.

I have the same problem that for some of my podcasts, chapters are not visible, although I can see them in Pocket Casts, Apple Podcasts and also with ffprobe when checking the m4a file.

As the hamster said above if the podcast creator / distributor uses software that creates non-standard chapters AntennaPod cannot see them.

Lets use the Bloomberg podcast above as an example.

( Tested with ffprobe & AntennaPod v2.7.1 )

Chapters show up when you grab the file from

Apple Podcasts
omny.fm
Google Podcasts

But not from

[bloomberg.com](https://www.bloomberg.com/news/audio/2023-04-06/imfffprobe -v-western-alliance-etfs-and-kurodanomics-podcast)
or Sticher

My hot take : The advertising networks that insert custom adverts on the fly are mangling the file en-route to the listener. Sticher is a good example, when tested from my IP, got chapters, when tested from a vpn chapters disappeared, over tor chapters came back, tested from a second VPN provider chapters disappeared.

@ByteHamster please feel free to close this issue as you have done your part, and the chapter mess is happening somewhere else

[ByteHamster]

We support a bunch of chapter specifications, but not yet the one most commonly used in m4a files. If you can see chapters in m4a files, that's probably because the author encoded them with ID3 tags (which is apparently not recommended with m4a). So this issue is still open

[dstapp]

I did some digging and found sannies/mp4parser to fetch metadata from the container, as shown here. I assume this would also be usable for chapters (not exactly sure how they are encoded in the container though): https://github.com/sannies/mp4parser/blob/master/examples/src/main/java/org/mp4parser/examples/metadata/MetaDataRead.java.

Unfortunately I'm not an Android dev so I didn't test that, but if if helps, once I find some more spare time, I can see if I can provide a PoC that works with the referenced files.

There's also an old issue for Voice, the guys there apparently implemented that by parsing the ffprobe output: PaulWoitaschek/Voice#205 (see followup / PR at the end of the thread).

Maybe the inclusion could be a feasible option if acceptable license/size wise?

[ByteHamster]

That library feels a bit like an overkill, doing way more than we ever need. I think a small class that does the parsing should be enough.

[HFPasfho]

This usually means that the podcast creator messed something up with the chapters (AntennaPod uses fewer workarounds than ffprobe), but feel free to create a new issue describing the problem in more detail.

@ByteHamster I don't think it's a good attitude. Nobody actually does chapters by hand so some popular programs must create them like that since none of podcasts with chapters I listen is supported by AntennaPod (that's why I created a feature request). And they are supported by other players or by ffprobe itself. I don't negate that AntennaPod supports some kind of kosher chapter style but the thing is it doesn't support many other types of chapters that are out there in the wild. I think AntennaPod as a tool should support as much chapter styles as possible or use some kind of library specialized in this task. Please don't be so orthodox that AntennaPod supports few true chapter formats and whoever uses software that produces others is wrong. I know that lack of standardization can be frustrating but that's how the world works. You have to adopt. Even if you are a hobby project (which is awesome but mainly for checking updates and downloading episodes because I actually have to use separate player supporting chapters and rewinding tracks using headset button).

[HFPasfho]

We support a bunch of chapter specifications, but not yet the one most commonly used in m4a files. If you can see chapters in m4a files, that's probably because the author encoded them with ID3 tags (which is apparently not recommended with m4a). So this issue is still open

@ByteHamster I hope it will be implemented soon because it looks like that's the most popular chapter format out there (which is odd that it's implemented as last)