Ensures album view orders albums by album artist

[andreas-henning]

Purpose

The order of albums in the album view is partially dependent on the filesystem from which the albums were read, i.e. if audio files are stored in the folders 'A' and 'B', then all albums in 'A' are listed before those in 'B'.

This pull request ensures albums in the album view are always sorted by album artist, irrespective of folder structure.

I mentioned this in a forum post (here).

Consequences

For the majority of users, who's album folders all reside under a single root folder, these changes will make no difference. It affects only those with a more complicated folder structure. It can negatively affect those who (mis)used the filesystem specifically to influence album sorting in the Volumio UI.

Implementation:

For the 'Albums' view I opted for the Timsort algorithm for performance reasons, in particular because it is optimally suited to handling partially sorted lists, which the album array (as produced by mpd) is.

This is noticeably faster than JavaScript's array.sort() and much faster than sortOn(). From my own tests running on a RPi3 sorting 1452 albums (a fourth of my collection):

sortOn = 0.296 seconds
timSort = 0.053 seconds

In light of this, and because I couldn't find anything that actually used "sort-on", I removed it from packages.json.

Installation

Deploy source to Volumio host, then:

cd /volumio
npm remove sort-on
npm install                                             # installs timsort

Timsort has a developer dependency on mdns which requires the file dns_sd.h. This is already included in the Volumio image for the RPi. Should it be missing on someone's custom Volumio host, it is distributed as part of libdns-dev:

sudo apt-get install libavahi-compat-libdnssd-dev    	# Debian, Ubuntu
sudo dnf install avahi-compat-libdns_sd-devel	        # Fedora, RHEL, CentOS

Update (12.1.2018):

Stumbling blocks when sorting lists in genre and artist views

Ideally, albums and tracks would be ordered identically in all views. However, sorting albums and tracks in the genre or artist views introduces some issues:

• In the artist and genre views, tracks are sorted (ignoring file system influences) by album artist AND album name AND disk # AND track #. Due to the lack of a disk #, there is no way to sort tracks in an identical way. Therefore, this PR makes no attempt to make track sort order independent of the file system, as that would change sorting behavior for everyone, including for people who have all their music in a single directory.
• When the genre view asks mpd for tracks, the response times can already be slow (particularly when thousands of tracks belong to one genre). Sorting those tracks again can make it unacceptably slow (sorting 2000 tracks in the genre view on an RPi3 adds about 2 seconds).
• Users can opt to have the track # displayed with the track name. This causes Volumio to internally concatenate the track # and name, making it clunky to work with either individually. Ideally, track# and name would only be concatenated in the UI. The back-end should keep these fields separate and independent of UI display/formatting considerations. This would allow for a much cleaner sorting implementation. Also, displaying track # will cause issues for tracks that lack a title, resulting in the UI showing "undefined" in some areas. (see issue)

Sorting all lists in all views in a consistent way requires the above issues to be fixed first.

[theinfinityproject]

It can negatively affect those who (mis)used the filesystem specifically to influence album sorting in the Volumio UI.

The fact that albumview reflects the user file sorting isnt a bug, its a helpful feature imho, specially for large collections. May i ask if its possible to implement some sort of settings which lets the user keep the sorting based on his/her folder structure.

[andreas-henning]

@theinfinityproject
Maybe you could explain your requirement better, as I don't see it.

If you want to browse your collection by folder structure, particularly if you have a large music collection like we both do, then the 'Music Library' view is exactly what you are looking for. From there you can navigate the folder structure much faster than in the 'Albums' view. Why is that not much better for your use case?

In the 'Albums' view:

• getting to the contents of directory C require you to first scroll past the contents of directory A and B (which in large collections takes forever).
• there is no way to quickly navigate between the contents of folders.
• makes no attempt to even visualize where the contents of one folder ends and another begins.

The 'Albums' view just isn't designed to do what you want in a user-friendly fashion, nor should it be IMHO.

IMHO adding a setting to maintain the current behavior is just adding a redundant feature to the 'Albums' view that is already solved better elsewhere.

Your thoughts?

[theinfinityproject]

First of i agree with almost everything u said and im strongly reminded of this

Workflow

That's being said, my library is sorted like this Genre(based on my criteria/taste)/Artist/(year)Album. While almost every sorting in my library is done via autotagger (beet), Genre isnt. Since it produce unwanted results like the following for example, Artist 1 has album A,B but album A is considered Rock and Album B Stoner Rock and therefore both albums would be in seperate folders. Thats why Genre sorting is done by myself manually.

With the current album view and the applied sorting system i get the following results: Albums are sorted via their Genre folders, Artist alphabetically and there respective albums in a chronology order. My first reaction was the same like urs, its a bug, since is expected Album view to be sorted A-Z, but after a short while i came to the conclusion that its actually a really nice feature.

Genre Rock (Folder Rock) N to P based on my folder structure in the album view

U mentioned Media Library view, but it lacks the beauty of scrolling thro covers (on a phone or tablet for example) while still being within my defined folder structure based sorting system. U could argue now that Genre view is that what i rly want but consider that Genre view is based on metadata and as i mentioned before genre metadata based on libraries like last.fm is completely useless imho. So my only option would be to tag the metadata manually if u would remove the old system.

Since most users are using one root folder anyway and wouldnt be affected by this change, why disrupt the "workflow" of those who took the time and created a more complicated folder structure and find it useful that this structure is reflected in the album view.

[andreas-henning]

@theinfinityproject

Thanks for elaborating! I see your point.

As I understand it, the 'Music Library' is technically what you are looking for. It just lacks album art. That is strange, because my 'Music Library' contains the exact same covers and layout options (grid/list) as the 'Albums' view, also on my Android phone. It's just as beautiful, just with folder based filtering. Yours really doesn't have album art? If that's the case, I'd say that is the problem we need to solve.

This is the thing: People's folder structures serve different purposes. In our household for example, each member has the music they own in their own folder. If I'm looking for my own music, I use the 'Music Library' view. If I want to browse all the albums we own as a group, I'd appreciate a list of albums that is sorted CONSISTENTLY across the entire collection.

In general, I'd say that browsing and sorting is something that should be entirely in the domain of the UI, i.e. sorting preferences should not impose any restrictions on how we can or can't organize our music collection on the file system. Tying the two together, as volumio does now, sacrifices a lot of flexibility, because we are no longer free to organize the filesystem independently of sorting/presentation considerations.

Does this change your mind about anything?

[andreas-henning]

If this were all up to me (which it isn't) this is what I'd say:

The whole point of the Albums, Artists and Genre views is specifically to allow us to browse our collections in ways that are independent of the file system.

File system based browsing is what the Media Library view is for. Period. If we really need the exact same file system based layout that we currently have in the Albums view, then this should really be implemented as an additional layout option in the 'Music Library':

hierarchical/flat + grid/list

[volumio]

@andreas-henning thanks for the PR, this actually solves what I also see as a "misbehaviour" of sorting.
While I understand the points made by @theinfinityproject I think we should be totally detached from FS while browsing by album, artist or genre...

That being said, I'm very busy ATM so checking & merging will take a while...

[theinfinityproject]

As I understand it, the 'Music Library' is technically what you are looking for. It just lacks album art. That is strange, because my 'Music Library' contains the exact same covers and layout options (grid/list) as the 'Albums' view, also on my Android phone. It's just as beautiful, just with folder based filtering. Yours really doesn't have album art? If that's the case, I'd say that is the problem we need to solve.

It does, thats not my point. When i come home ill take a bunch of screenshots. Then its easier to see the difference.

I'd appreciate a list of albums that is sorted CONSISTENTLY across the entire collection.

I agree. But ur approach dosnt achieve that actually. Let me explain what i mean by pointing smth out we havnt talked about yet. The default behaviour in volumio as in most file system is to sort alphabetically. So as we open the media library view, we see folders sorted alphabetically, if we open Artist view, we see artists sorted alphabetically and the same is true for the Genre view. Ur proposal tho would differ from this, as it would sort albums within the Album view alphabetically based on the tag albumartist and not based on the tag albumname. The latter tho is imho the consistent approach, isnt it?

So id like to opt for more flexibility and propose the following: Why not implement a simple Dropdown Menu within the Album view with the following options:

Sort via

• albumname
• albumartist
• folder based
• year

This would satisfy the request made by @volumio to be independent from FS while still increasing user flexibility and thus improving user experience.

[andreas-henning]

It does, that's not my point. When i come home ill take a bunch of screenshots. Then its easier to see the difference.

This is what I now think you are saying (rephrased in my own terms):

---

Functionally, the 'Music Library' provides what I require. From a usability perspective it is in fact the superior solution. However, I prefer the 'Albums' view for aesthetic reasons, as it omits hierarchical folder navigation and instantly displays album art.

What I'm looking for is the ability to maintain my folder based sorting without introducing any grouping and/or hierarchical navigation.

---

If that's correct, I think that is how your requirement should be stated.

However, I suspect there is still something of a misconception here. I suspect that to your mind, the 'Albums' view is primarily about getting straight to the album art, and that's why you want the option to sort by folder in that view. Although I'm anything but an authority on this, I'd say that's likely an incomplete understanding:

... as we open the media library view, we see folders sorted alphabetically, if we open Artist view, we see artists sorted alphabetically and the same is true for the Genre view. Ur proposal tho would differ from this, as it would sort albums within the Album view alphabetically based on the tag albumartist and not based on the tag albumname. The latter tho is imho the consistent approach, isnt it?

No. Grouping is not the same as sorting.

Music Library = g(folder) / s(album artist) = hierarchically group by folder, sort albums by album artist
Albums = s(album artist) = sort albums by album artist
Genres = g(genre) / s(album artist) = group by genre, sort albums by album artist
Artists = g(artist) /s(album artist) = group by artist, sort albums by album artist

Most views have an additional grouping. The groups represent filter options. After the filter is applied, the section of the results that displays albums are sorted by album artist (or at least that seems to be the intent). The 'Albums' view is the only view without a grouping, so there we get straight to the results, sorted by album artist.

Because every list of albums is sorted in the exact same way, by album artist, sorting is in fact consistent across all views.

My pull request ensures that sorting of albums is also consistent across installations, i.e. that people with more complicated folder structures have the same (and for most people actually useful) sorting experience as people with a single root folder for all their music.

Either way, we completely agree that sorting is something users should be able to influence via a dropdown box (for every list, not just in the 'Albums' view). I've also talked with people who only listen to classical music, and would therefore prefer all album lists to be sorted by album name. A dropdown would allow for something like that too. We just don't have that (yet?). If volumio decides to integrate my pull request I might take a shot at that feature later on.

Anyway, I hope I understood your requirement correctly now. If volumio accepts my pull request I'm sorry to say it likely will screw things up a bit for you. On the other hand, I think you'll remain better off than we are now. You will at least still have a way to find what your looking for. You'll just have to click once more in the 'Music Library' view, whereas many of us with more complicated folder structures currently have no way of systematically searching for albums at all, as views just aren't sorted in ways that are useful to us.

I say we let volumio decide what happens now and then go from there.

[commentsonly]

Andreas, could you explain on how to ensure that 7 different release of the very same album will be kept separated from each other if simply assuming that one could sort by an albumtitle?

It doesn't work, as long as you accept that the metadata designed to told the albumtitle does get filled with the content which is the albumtitle, not with additioanl things which have to go in differnt metadataframes exclusivly being defined for such things such as albumversion or albumsubtitle

MinimServer is amongst the very few pieces of software I'm aware of which has a dead stunning implementation to cover such topics. Since it differs for content between what's displayed and what's used for sorting.
Allowing to handle for example 5 releases of an album using the very same albumtitle, while sorting / and the ability to not accidently merge these all into 1 album) gets done by defining the sorting using more metadata such as album.sorttags={albumartist, data, releaseyear, album, albumsubtitle}

But as long as softare such as volumio, runeaudio, moode and others are limited to just a handfull of metadataframe (as it was way back in the 90th when using windows) you can't expect that sorting besides the physical structure would work. Perhaps with 50 albums, perhaps with 500 but not you collection passed 100K titels or more.

So in case the Volumio software gets user sorting capabilities, please ensure that 1 sorting criteria remains which is "sort by physical structure".

[andreas-henning]

@commentsonly
You make some very good points I had not yet considered. Thank you.

Ideally, this would work just as well as in minimserver. A few questions:

1
Could you tell us which ID3 frames (according to the 2.4 spec) minimserver is using for album-version and album-subtitle?

2
How are those fields used? I assume the fields album-name, album-artist, album-subtitle and album-version must all be identical for minimserver to consider the tracks as belonging to the same album. Correct?

3
If Volumio also considered those fields in that way, would you still require the ability to sort albums by folder structure? I'm not sure this is even possible with mpd, I'm just asking to make sure I understand.

[volumio]

Just tested this, and indeed performance is very good.
To have the sweet spot of usability and features, before merging I will add an option to sort on file structure, album or artist.
Thanks for your patience guys...

[commentsonly]

4 Andreas:

1. http://minimserver.com/ug-library.html see the section TAG mappings at the very end.
   If you need more adivce on that (not Volumio related) send a pm.

2. metadata means you embed informations into files. If you don't, no information, and since so, no way to make use of it.

3. I'm a big fan of metadata, since too often I lost some information due to accidents such as:
   I moved the audio files from one dir into another forgetting about the artist pic in the parent folder of the albums and the albumpics within each album folder.
   Lessons learned ..... unembedded information is equal to no information since you loose if too often.
   And if you loose the audio file, you won't need the metadata, whereas if you still got the audio file but lost the metadata it's a pain.

At the moment I could rename all my audio files similar to a database, from 1.flac till 100000.flac and I could even place these all into 1 big folder.
If the software is up to in (and I got some such as Minimserver) Volumio would see them all separated into aritsts and it's albums without a problem. I think that's the idea behind using embedded metadata.
And yes for some special cases it's also essential you're able to extract embedded information.

Just noticed that Volumio does display aritst art if being placed in the artist folder if you have something such as <collection<aritst><albums>\ titles.
took me 2-3 minutes to populate the artists folders with the embedded artist images.

For volumio I would have liked a similar ability such as what MinimServer offers.
The ability to define how things are displayed (not server source, but mounted sources).
So that I could define:
album.displaymode=albumtitle [albumsubititle] (albumversion)
and/or
title=title [subtitle] (remix)
since building representation based upon metadata, best bet differnt metadatafields is much more flexible but asking poeple to re-tag their collection to the needs of 1 piece of software. It's pretty unlikely one only uses 1 piece of software and it's as unlikely 2 or 3 software titles to behave the very same way.

[andreas-henning]

@commentsonly

1. Okay, I see the tag used for album-subtitle (TSST). I still don't see the tag you referred to as album-version. Can you clarify which tag it is you're referring to here?
2. You misunderstood my question. My original view was that album-name and album-artist were the only fields required to determine which tracks belong to the same album. You pointed out that isn't true, as some people may own multiple releases of the same album. So, would you agree that all tracks that contain the same value in their album-name (TALB), album-artist (TPE2), album-subtitle (TSST) and album-version (?) fields belong to the same album?
3. I'll take that as a NO (sorting by folder structure would then no longer be necessary) ;-)

I've noted the feature request to define how items are displayed/labeled.

[commentsonly]

1. If you define that your tagger does a frame TXXX ALBUMVERSION it'll work similar to all pre-defined standard frames. And if your tagger is up to it it won't simply show you a list of TXXX frames one after the other in case you defined more but one .. but does handle these TXXX frames as it does handle TALB and others, where it shows you Album: and not TALB:

2. Dead easy, open discogs.com and search for "Abba - Arrival" you'll see that there's 207 versions of this album at the moment.
   Now ask yourself how many items of metadata you would need to get these separated from each other. I'm pretty sure that albumartist + albumtitle won't work. Granted I not aware anyone does collect all available releases of one album, but I know many which have a 24/96 vinyl besides a 24/96 sacd remaster besides a 16/44.1 expanded deluxe edition, since in most cases these all sound differnt and some have extra tracks.
   To get these separated you either need release dates beside name and albumtitle or you need some more metadata.
   EAN/GTIN numbers are pretty good to separate things since they are unique .... drawback, older releases don't have such things.

3. The point isn' if it could be removed or not, the question is about what's the drawbacks and what's the benefits if being removed. I'd say no benefit but drawbacks for the ones with inaccurate/incomplete metadata. Not to mention that you need to give names to directories anyway, means if you won't name the top level so that it matches the artist name and below a level what names would you use for these directories?

[andreas-henning]

@commentsonly
We're somewhat speaking past each other here. You're providing the answers I already know rather than those I'd actually want.

1. Okay. It's a custom ID3 field. That's all I wanted to know. Everything else follows from that and is then obvious.

2. Once more. In your view, What ID3 fields, standard or otherwise, should an application understand/support (and why) so it can handle all the ways you can foresee, in which people may want to determine which tracks belong to which album. I don't care about solutions. I'm trying to ensure I understand all the problems/issues people may be facing. EAN numbers would be an example of such a field that wasn't yet mentioned.

3. The question was: for your specific use case, if the mentioned ID3 fields were considered in determining which tracks belong to which album, would sorting albums by folder still serve a purpose in the context of your problem? For you personally. Again, I currently don't care what either of us thinks a solution should look like. I'm trying to ensure I really understood your problem.

[commentsonly]

1. yep
2. good software comes with a so called mapping-table linking internal names to id3v2 tagids, vorbiscomments and such like so that the user could adjust that in case of need.
3. no problems since I use mininserver which does help me along the other 90% of software with limited capabilities.

[andreas-henning]

You're a master of evasion :-) Let's leave it at that.

[commentsonly]

Not at all, but you're welcome to re-read what was posted.

[andreas-henning]

There's a lot of going-off-on-tangents that doesn't tell me anything I don't already know. You occasionally touch on potential solutions, which I find interesting, but isn't what I'm trying to understand. There is very little discussion of the actual problem, so there is nothing worth re-reading.

[andreas-henning]

Any ETA on this @volumio? Would be much appreciated! Thx!

[andreas-henning]

@volumio
Well, it's been a year. What's the current situation?