Music loading issues

[OxygenCobalt]

The android media APIs are dying a slow, painful death. This is a super-issue to document the myriad of unfixable issues that have been encountered with media files in Auxio.

These are the issues I am aware of.

1. Genre indexing is slow and inefficient
2. Poor ID3v2.4 support [No dates, weird artists]
3. Poor FLAC support [No dates]
4. Literally any UTF-8 tag with non-ASCII characters will look corrupted
5. An uncountable amount of OEM-specific insanity that I can't even include

None of these are fixable because either it's google's job, and they could not care less. What I'm very likely going to do is add SoundPod support and focus most development on that instead of the sinking ship that is the android filesystem.

[OxygenCobalt]

I've done a refactor to the music loader for the next Auxio release.

The new system is a bit slower, but now it has album artist support! Auxio will now always use the Album Artist tag when it comes to grouping artists, defaulting to the normal Artist tag when it isn't present. This system isn't exactly ideal, as it will ignore any collaborator information and only show the album artist, but it's still an improvement from a bunch of fragmented artists.

This change also fixes the artist mis-assignment issue that occurs on some Samsung devices with crippled MediaStore tables.

[itsmusictomyears]

Just chiming in to say #3 does indeed cause problems with large libraries: Auxio 1.4.2 takes about 55 seconds to get up and running with my library of 29.000 tracks. Everything's fine once it's all loaded up, though.

[OxygenCobalt]

Yeah, I figured. The problem is both systems make tradeoffs.

Loading everything on the fly results in a fast startup time, but really slow operation time whenever you want to shuffle all songs and results in problems with properly restoring things

Loading everything at once takes really long at the beginning and maybe risks an OOM if a library is big enough, but seems to work pretty well in everything else.

Personally, if my library grows to the size where it becomes a problem loading it in, I'll likely create some flavor of Auxio with SoundPod support [or something similar] to stop this issue.

[itsmusictomyears]

Would it be possible to create a custom indexer, bypassing the native one? I don't know whether it would fix the startup times, but it would probably be able to do a better job at reading the tags than the native indexer.

[OxygenCobalt]

Something I may do in a future update is split my media indexer into three modes a la Simple Gallery:

• Correct: This is the media indexer Auxio will use in 2.0.0. Index only the song database with full album artist/genre support. Good for well-tagged libraries, but is much slower.
• Compromise: Effectively the media indexer from 1.4.2. Support genres, but split queries across the song and album database for speed and only rely on the artist field.
• Speed: Like compromise, but drop pre-sorting and genre support as well [which is a huge bottleneck if you've read my massive screed in the MusicLoader class]. This is the best for large libraries, but at the cost of good metadata support.

[itsmusictomyears]

I don't know if you've made any big changes to the indexing yet, but I just noticed in v2.0.1 that the song count in the library, at least on the genre tab, is often way off. For example, the library says I have 2 songs for the genre 'Power Electronics', but there's actually 24 songs. However, for some genres the count is correct.

There are also genres with 0 songs, and indeed no songs show up at all. I checked, and I don't have any music files belonging to those genres on my device. However, I think I had some music of those genres on the device in the past, so it would seem some cached stuff is not getting cleared.

Additionally, it appears that the indexer also struggles with certain special characters, displaying them as completely different characters. Sometimes it only happens for some tracks on an album (with the album title glitched) whereas other tracks on the album are fine (with the album title displayed the way it should).

Maybe all of these problems are Google problems :)

[OxygenCobalt]

Auxio 2.0.(0/1) uses a new music loader that only indexes songs and genres and creates everything based off of that. I imagined that it would fix a lot of the strange OEM-specific indexing issues, as previously I would query the less reliable Album database for speed.

For example, the library says I have 2 songs for the genre 'Power Electronics', but there's actually 24 songs. However, for some genres the count is correct.

That is extremely weird. I get the amount of songs based off of the list of songs I attach to each genre. That means that somehow the list of songs is not being truthful about it's size. I imagine that it might be Auxio buckling under the size of your music library, as the genre loading process is basically repeatedly iterating through the entire list of songs repeatedly to link each song with their genre. Not sure what to do here.

There are also genres with 0 songs, and indeed no songs show up at all. I checked, and I don't have any music files belonging to those genres on my device. However, I think I had some music of those genres on the device in the past, so it would seem some cached stuff is not getting cleared.

Completely a caching issue. I could add another check to the music loader that culls empty genres to fix this though.

Additionally, it appears that the indexer also struggles with certain special characters, displaying them as completely different characters. Sometimes it only happens for some tracks on an album (with the album title glitched) whereas other tracks on the album are fine (with the album title displayed the way it should).

Seems like an encoding issue. ID3 is notoriously finicky with encodings and I wouldn't be surprised if googles ancient metadata parser is trying to decode UTF-16 without handling surrogate pairs (or other wacky Unicode nonsense).

I am working on introducing loader customization in Auxio 2.1.0. You'll be able to revert to something resembling the old 1.4.2 system, and you'll also be able to disable genres entirely, which will likely half your loading times if my benchmarks are correct.

[itsmusictomyears]

Oh, perhaps I made myself misunderstood, I think the new way is actually better than before - I'd say it's faster, or at least not slower, than before, and the fact that the main UI now shows up immediately is certainly an improvement, too - so I don't think there's any need to revert to the old behaviour.

One thing I do wonder about, though, is if doing the indexing every time the app is loaded (at least once every boot) is not straining the battery? On devices with a big battery and a small library, it's probably negligible either way, but does it make sense on devices with a big library and a small battery (like mine)? I genuinely don't know if it could or does impact battery life. If it does, could you add an option to disable the automatic (re)indexing on boot, keep the information from the previous indexing cached, and only reindex when explicitly selected (or after a certain amount of time has passed, or after an update to Auxio has been installed, ...)?

[OxygenCobalt]

Oh, perhaps I made myself misunderstood, I think the new way is actually better than before - I'd say it's faster, or at least not slower, than before, and the fact that the main UI now shows up immediately is certainly an improvement,

I was more arguing that being able to swap back to the old system would help solve some of the strange metadata issues. Through if the new system works well for large libraries like yours, I'll just shelve the idea of customizing the music loader until there's demand for it.

One thing I do wonder about, though, is if doing the indexing every time the app is loaded (at least once every boot) is not straining the battery?

Auxio re-indexes music every time it has a "cold start", which basically means when you open the app for the first time after a reboot or after the phone kills it to save battery. The android system does most of the heavy lifting here, and Auxio just uses whatever it's given, so I'd imagine that it's not too intensive.

f it does, could you add an option to disable the automatic (re)indexing on boot, keep the information from the previous indexing cached, and only reindex when explicitly selected (or after a certain amount of time has passed, or after an update to Auxio has been installed, ...)?

The android system already does that. It's just that querying the media database takes so long. All caching on my end would do is make Auxio more unstable, which is something I don't want.

[itsmusictomyears]

Thanks, I think I understand now!

[ghost]

This problem is present in all audio players on Android except VLC on Android. But there is also an issue with ö, ü, ä which are partially displayed weird (only FLAC, not with MP3s).

LineageOS 18.1 with Auxio:

Gentoo Linux with GNOME and Lollypop

[OxygenCobalt]

Oh. I know what's going on there.

FLAC titles are encoded using UTF-8, so the umlaut u is encoded as the bytes "C3 BC", where C3 is just a lowercase u and BC is the special character that adds the two dots to make it an umlaut.

What android is doing is that it's treating that album title metadata as if it was encoded in Latin-1, which is a much older and more restricted character set that isn't outlined the FLAC specification at all. In this case, the "C3 BC" bytes will be individually interpreted as the characters à and ¼.

MP3 files don't have this issue since they use a different metadata format that explicitly specifies the encoding. The way VLC gets around this is by parsing the metadata themselves, but that is a nightmare to implement and would ruin Auxio's user experience. Theoretically, I could remedy this by parsing the albums database instead of the songs database for album names so that they are encoded correctly, but that causes even more problems since OEM's seemingly break that database on a whim.

There is nothing I can do about this since it's a flaw within Android's ancient metadata parser. You might want to take this to the LineageOS devs, but even then I'm not sure how much they can do since the metadata parser is part of the upstream android project. This problem is so wide-spread that I'm considering just submitting a patch to AOSP myself that tries to fix some of this insanity.

[132ikl]

Are there any plans for the new loader to be able to read dates from FLAC files?

[OxygenCobalt]

Android's metadata parser doesn't read dates from any file that is not an MP3 audio with ID3v2.3 metadata. This issue has been around for over a decade. There is nothing I can do about it.

Also, I already finished the new loader, it was released with Auxio 2.0.0. Currently the loader is in the best place it can be and I have no plans to change it.

[OxygenCobalt]

Confirmed that with the current loader, every single metadata tag that's in UTF-8 will have any non-ASCII characters corrupted. This is probably because during the conversion to a java string, it tries to dynamically determine the encoding. This usually ends up with it choosing Latin-1, Shift JIS, or UTF-16 for backwards compatibility, because god forbid breaking some 25-year-old business application.

As for what I recommend, just rewrite your tags. Convert accented characters to their phonetic forms. Convert symbols to their names or similar. I don't see anything that can be done about this that doesn't involve creating my own metadata parser, and at that point I may as well go "screw it" and just add SoundPod support instead of trying to salvage the dying android file APIs.

[OxygenCobalt]

@Sabonk Seems like it's specific to external storage. I'm spinning this off into #84.

[ygregw]

Whenever I adb push new music files to my phone, running lineage os 18.1, auxio is not able to detect them upon restarting, unless I hackily/clumsily rename the pushed folders on my phone. Is this yet another eccentric annoyance of Android's filesystem?

[OxygenCobalt]

Kind of, @ygregw. It seems like your issue is composed of two smaller issues:

1. Making Auxio rescan music on it's own. This is being worked on under Automatic rescanning #72, however it will likely require a constantly-running background task in order to automatically detect music in a way that newer versions of android will accept. In the mean time, you can try the Reload Music option in settings.
2. Getting Android to reindex the new music. This is a system issue and I'm not sure how much Auxio can mitigate that.

[ygregw]

I don't think issue 1 is pertinent since new folders are not detected even after Auxio is killed and restarted. Somehow it seems adb-pushed folders are not visible to Auxio until they are renamed. This issue can be reproduced for another music player, Simple Music Player, in F-Droid which is why I suspect it's not an Auxio bug.

On a side note, I actually quite like the fact that there is no constant and wasteful rescanning on the background especially because my music repository tends to be rather stable and subject to very infrequent changes.

[ygregw]

Issue solved by using Syncthing to add and tag music files. Friendship ends with adb push and pull...
Thanks a lot for all your work developing this great program!

[OxygenCobalt]

Update: I've done some work on parallelizing a possible manual metadata parser and have been able to get throughputs of ~16ms per song loaded, which is in line with other apps. It also has the bonus of not having to bring in any redundant metadata libraries that don't play along with android, as the whole system relies on ExoPlayer's metadata system. This is actually really exciting, as I can eliminate much of the "unfixable" metadata issues by parsing metadata myself. No idea when this might land in Auxio, as it was just a prototype.

[OxygenCobalt]

Update: Dates will actually work on FLAC/Vorbis files as long as you use the YEAR field instead of DATE. Again, this is because android's metadata parser has not been updated in upwards of 10-15 years.

[hakmad]

I'm not sure if this is the right place for this or if I should create a new issue, but I have a problem loading some music on my device when using Auxio: specifically, Auxio doesn't load newly downloaded music.

Expected behaviour:

1. Specify folder to load music from in Auxio (e.g. the Music folder)
2. Download music as an MP3 file (e.g. use youtube-dl and Termux) to above folder
3. Select "Reload Music" from "Settings"
4. Music should now appear in list of music

To reproduce: perform steps 1 to 3 above. Auxio (and the default music app) refuse to load new music in this way. Another music app, Vinyl Music Player, does load new music properly.

Strangely enough, selecting the reload option in Vinyl ("Scan Media" from the sidebar) also causes Auxio and the default app to load the music correctly.

I have no idea why either of these happen. Sorry if this is the wrong place to put this issue, I'm not very good at this haha. 😅

[OxygenCobalt]

This is tracked by #72 @hakmad. Basically, reloading music like that is a bit of a difficult technical problem, especially when you do not query the database in the UI but instead create an in-memory representation of the music library (Auxio is somewhat forced to do this due to all of the QoL improvements I have made). I am working on it though.

[hakmad]

Ah, sorry! I didn't realise there was an issue for this already, that's my bad.

[OxygenCobalt]

#72 is now in Auxio as an experimental option called "Automatic reloading". It's not like the behavior that you see in other music players (It will show a persistent notification), but this is to enable much more reliable music updates.

@hakmad @Naverim

[animaldaydream]

I think this is a better place to track these rather than opening new issues, but I thought I'd ask anyways.

I use Quod Libet on my PC and have disambiguation provided by MusicBrainz UUIDs that are stored in the tags when you use a compatible tagger like MusicBrainz Picard.

When there's two artists with the same name, or two different releases with the same artist and album tag, it successfully disambiguates them and offers them separately, with different artwork, release date, and so on.

Second, Picard also stores in the tags the first release date of a release group, which means all releases of an album will have both the original release date, and the release date of that particular album.

Third, Picard and many other audio taggers and metadata standards support storing multiple values in a single field. Picard stores multiple artist values in the "artists" field (mind the 's'), and stores them concatenated as credited officially in the "artist" field, among others.

I currently set up my library to contain multiple values in the regular Artist and Album Artist fields, and they're standardised too so different spellings on different releases are all combined into one. Thing is, Auxio only takes the first tag it finds, and apparently, it's the first tag that it finds in the first file found that contains it.

In this case, it found the "Björk" tag in an earlier file, and on an album with both the tags "Dirty Projectors" and "Björk" (in that order), it completely discarded the tag "Dirty Projectors" on the release.

In summary:

1. Consider using MusicBrainz IDs if present, so that artists and releases are disambiguated and fused properly.
2. Some other tags provided by Picard could be useful to further sort and group the albums of a particular artist as the user chooses.
3. Currently, when a file has multiple artist/albumartist tags, only one is used and the rest is discarded.

[OxygenCobalt]

Okay, a lot to unpack @animaldaydream.

Consider using MusicBrainz IDs if present, so that artists and releases are disambiguated and fused properly.

While this is technically possible with #128 now, the implementation specifics would be a bit weird. Auxio sort of replicates an ID by hashing metadata together, and theoretically I could just throw in the MusicBrainz ID into the hash calculation. However, this comes with a major issue: I can only extract MusicBrainz IDs using #128, and that setting can be toggled. I want to make sure that the hashes stay relatively consistent to reduce playback state wipes, but adding such a tag would completely break that if one were to switch from one setting to the other (or vice versa).

I think I would re-consider it if #128 graduates from experimental setting to Auxio's default, unchangeable behavior. Then, the tags I can only extract with it can be thrown into the hash calculation without issue. Depends on performance and memory usage on a wider scale.

Some other tags provided by Picard could be useful to further sort and group the albums of a particular artist as the user chooses.

I think this is already supported by #128, assuming you are talking about Original Years (TORY/TDOR for mp3, ORIGINALDATE for opus/vorbis/flac`). Fine-grained dates (YYYY-MM-DD HH:MM:SS) and release types are also supported with #128, if you like those.

Currently, when a file has multiple artist/albumartist tags, only one is used and the rest is discarded.

This is intentional behavior. Personally, I add collaborator information to the "artist" field, and then choose the most major collaborator in the "album artist" field, and so I designed Auxio with that in mind. To make songs correspond to multiple artists is a technical nightmare both internally (All music is in-memory, so I need to keep the the library data-structure simple to lower memory usage), and in the UI (Navigation in particular would be completely broken, as "go to artist" could apply to one of several artists).

[OxygenCobalt]

As for this thread, I'm planning to retire it when #128 is finally released. This was originally meant as a dumping ground for insane issues with MediaStore, but #128 solves many of those issues, and it's likely Auxio's future anyway. Hence, standalone issues will be more appropriate.

[animaldaydream]

This is intentional behavior. Personally, I add collaborator information to the "artist" field, and then choose the most major collaborator in the "album artist" field, and so I designed Auxio with that in mind.

Man, I'll be honest, this is extremely weird and unexpected behaviour. I'd rather all the artists were completely concatenated. Which I'm gonna do myself with a Picard script instead. [By the way, that particular album has the same two artists on both the 'album artist' and 'artist' tags, on every track, and even still only one is shown.]

To make songs correspond to multiple artists is a technical nightmare both internally (All music is in-memory, so I need to keep the the library data-structure simple to lower memory usage), and in the UI (Navigation in particular would be completely broken, as "go to artist" could apply to one of several artists).

Well I don't know how that works, so I'll just believe you. Poweramp handles that fine though, I don't know how the dev does that, though it concatenates the values and then you have to separate them placing the hard-coded connector in a separate setting that splits them back up, etc.

Quod Libet handles that fine too, but I'm starting to nitpick.

I'll open a new issue though. I just need to collect my thoughts, I think.

[OxygenCobalt]

By the way, that particular album has the same two artists on both the 'album artist' and 'artist' tags, on every track, and even still only one is shown.

That's a lot weirder @animaldaydream. Can I have a sample file and the behavior you expect in particular? I swear, if android is being "helpful" again and mangling your artists I am going to lose it.

Poweramp handles that fine though, I don't know how the dev does that, though it concatenates the values and then you have to separate them placing the hard-coded connector in a separate setting that splits them back up, etc.

Then again, poweramp has a team of people behind it that can handle the many edge cases of such a setting (And force the android OS to do what they want), while I'm just one guy. Not saying I can't handle it, it's mostly navigation that is my major concern. Perhaps some other time.

[animaldaydream]

Then again, poweramp has a team of people behind it that can handle the many edge cases of such a setting (And force the android OS to do what they want), while I'm just one guy

Poweramp has exclusively one developer who refuses to allow anybody else to even look at the code :D

It's his job, though. He actually gets paid to do this. Even if his community hates it because resolving basic issues takes months and even years, while new features still come in.

I'm being annoying though. I'd help you, but all I have to offer is advice on how other 'decent' players handle this 'decently'. But believe me, from an UI standpoint, it's totally doable; some music players do pull this off as long as the tags are correct. I don't think open source devs would usually mind you taking a hint from their UIs, would they?

So long as it behaves, I'm open. I'm extremely annoyed at the current state of music players and I know exactly how an UI should handle this properly without compromises.

I'm gonna make a file tree that replicates this bug in a while, and I'll open a new issue afterwards.

[animaldaydream]

I swear, if android is being "helpful" again and mangling your artists I am going to lose it.

Actually, you still use MediaStore, right? Because this fork of Phonograph runs into the exact same issue.

https://github.com/chr56/Phonograph_Plus

[animaldaydream]

OK, here it is: #195

[OxygenCobalt]

2.6.0 is released, so this issue is bring retired. Please file new, individual issues with a particular problem.