New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sorting in paned browser #1785

Closed
pfps opened this Issue Jan 14, 2016 · 21 comments

Comments

Projects
None yet
4 participants
@pfps
Contributor

pfps commented Jan 14, 2016

In song lists the sort version of tags is used, so The Beatles will be sorted as Beatles, The for a song list column of artist and artist value of The Beatles and artistsort value of Beatles, The

The paned browser does not do this, but it should.

@pfps

This comment has been minimized.

Contributor

pfps commented Jan 15, 2016

Sorting of tags is not even particularly well defined for the paned browser. Consider a song with multiple artists. What should be provided for the artistsort value? Does anyone have a decent specification of what should happen even if there are different numbers of values for artist and artistsort?

This also affects sorting in the song list, but not as badly as multiple values are concatenated there.

@nodiscc

This comment has been minimized.

nodiscc commented Jan 15, 2016

Hi, I'd like to know more about this, but your initial problem is not very clear to me. Maybe you could post screenshots of the problematic sorting? What is artistsort? From what you describe I only understand that sorting is not consistent in paned browser (how?).

@pfps

This comment has been minimized.

Contributor

pfps commented Jan 15, 2016

Uploading quotlibet.jpg…

In the song list The Beatles is sorted before Chicago (because there is an artistsort tag of Beatles, The for all songs with The Beatles as artist). However, in the paned browser in the artist pane The Beatles is sorted without regards to the artistsort tag.

Hmm. I'm not seeing the attachment. I'm running my browser in maximum paranoia mode so maybe that is interfering with image upload. Anyway the relevant information from the screenshot is that in the artist pane in the paned browswer I see

...
Sting
The Beatles
The Rolling Stones
...

whereas in the artist column of the song list I see

...
The Beatles
Chicago
...

@pfps

This comment has been minimized.

Contributor

pfps commented Jan 15, 2016

@pfps

This comment has been minimized.

Contributor

pfps commented Jan 16, 2016

The next comment is a long discussion on a change to processing of tags that I think will much better support sorting in the paned browser (which uses the separated way of processing multiple values for tags).

@pfps

This comment has been minimized.

Contributor

pfps commented Jan 16, 2016

Sorting currently doesn't work very well in in some places in Quodlibet. Tag patterns didn't
use sort tags for sorting (artistsort for artist, etc.) until recently. The
paned browser doesn't use sort tags at all. Further, sort tags are
currently implemented in song lists by taking the text of the tag and
replacing tag names with the sort tag name. This is complex and currently
doesn't catch all cases, e.g., tied tags in tag patterns.

Sorting in the paned browser is going to be even more complex. Keeping
track of the correspondence between displayed values and sorting values is
nigh impossible using the current mechanisms. (Consider, for example, a
song with three artists but only two artist sort values. How are these to
be connected when all that is seen are the final values for separated tag
pattern processing?)

I think that it would be better to update the core processing of tags so
that the correspondence between a display value and a sort value can be
tracked through tag processing. This doesn't solve all the problems with
sort tags and multiple values but it at least lets a start position be
defined at the tag itself and then propagated through the tag composition
mechanisms. (One remaining problem is the dependence on ordering of values
between a tag and its associated sort tag. Quodlibet does not provide good
facilities for controlling the ordering of tag values during editing.)

Here is my proposal for handling sort tags in the core of tag processing.
See below for an example.

Conceptually, a tag or tied tag or tag pattern is a function from a song to
a set of results in the form of pairs of strings where the first is the
display string and the second is the sort string. The sort string is
directly suitable for sorting, i.e., any markup is removed before any final
results come out of tag processing. Tag results are computed in two
different ways, by concatenation (used, for example, in song displays and in
song lists) and by separation (used in the paned browser). For
concatenation there is only one result but for separation there can be
multiple results. Consumers of tag results initiate tag processing in
either a concatentation or a separation setting and only get to look at the
final results.

The primitive operation in tag processing is generating the results for a
single tag (e.g, artist or ~basename or ~performers) on a song. In a
concatenation situation the display string is the concatenation of the tag
values separated by ", " or an empty sting if there are no values and the
sort string is the same unless the tag has an associated sort tag in which
case the sort tag is used instead. In a separation setting the values for
the tag are determined. If there is an associated sort tag the values for
the sort tag are then determined, if any. The number of results is the
maximum of the size of these two with the i'th result consisting of the i'th
tag value and the i'th sort tag value. A missing value for either of the
tags causes the i'th value for the other tag to be used instead. If both
tags have no values then a single result of two empty strings is used.

Tied tags work in a concatenation setting by concatenating the appropriate
values from above with " - " as a separator. Tied tags work in a separation
setting by unioning the appropriate results. (This incorporate my proposal
for a change as to how tied tags work in a separation setting.)

Processing of tag patterns is done left to right, starting with a single
result of two empty strings. Conditions in tag patterns that are just a tag
check whether there are any values for the tag, ignoring any associated sort
tag. Strings in tag patterns are simply concatenated onto the end of all
strings in the results constructed so far. In a concatenation setting tags
or tied tags concatenate the tag or tied tag display string (sort string,
respectively) onto the display string (sort string, respectively)
constructed so far. In a separation setting tags or tied tags a result is
produced for each pair of the results constructed so far and the results for
the tag or tied tag evaluated in a separation setting. (So if there are
three results constructed so far and two results from the tag or tied tag
then six results are produced.) The display (sort, respectively) part of
the result is the concatenation of the two display (sort, respectively)
parts.

By the end of markup all display information is removed from the display
part of all results.

Example:

artist
The Beatles
Stingers

artistsort
Beatles, The

performer
John Lennon
Ringo Starr

performersort
Lennon, John
Starr, Ringo
Sting

Concatenation Setting
artist
{ < "The Beatles, Stingers", "Beatles, The" > }
performer
{ < "John Lennon, Ringo Starr", "Lennon, John, Starr, Ringo, Sting" > }
artistperformer
{ < "The Beatles, Stingers - John Lennon, Ringo Starr",
< "Beatles, The - Lennon, John, Starr, Ringo, Sting" > }
:
{ < "The Beatles, Stingers:John Lennon, Ringo Starr",
< "Beatles, The:Lennon, John, Starr, Ringo, Sting" > }

Separation Setting
artist
{ < "The Beatles","Beatles, The" > ,
< "Stingers","Stingers" > }
performer
{ < "John Lennon","Lennon, John" > ,
< "Ringo Starr","Starr, Ringo" > ,
< "Sting","Sting" > }
artistperformer
{ < "The Beatles","Beatles, The" > ,
< "Stingers","Stingers" > ,
< "John Lennon","Lennon, John" > ,
< "Ringo Starr","Starr, Ringo" > ,
< "Sting","Sting" > }
:
{ < "The Beatles:John Lennon","Beatles, The:Lennon, John" > ,
< "The Beatles:Ringo Starr","Beatles, The:Starr, Ringo" > ,
< "The Beatles:Sting","Beatles, The:Sting" > ,
< "Stingers:John Lennon","Stingers, John, Lennon" > ,
< "Stingers:Ringo Starr","Stingers:Starr, Ringo" > ,
< "Stingers:Sting","Stingers:Sting" > }

@lazka

This comment has been minimized.

Member

lazka commented Jan 16, 2016

One remaining issue is what to do about:

{ < "The Beatles","Beatles, The" > ,
< "The Beatles","" > }

@pfps

This comment has been minimized.

Contributor

pfps commented Jan 16, 2016

If you mean how to handle sort values that are empty strings, then these probably should be treated just as for missing sort values, i.e., the display value is used as the sort value.

If you mean how to handle display values that have several different sort values, for example when showing a pane, then that's already a problem and should be handled the way it is now.

@zsau

This comment has been minimized.

Contributor

zsau commented Jan 16, 2016

If I understand correctly, you're proposing that QL should associate sort tags with their base tags according to the order in which the tags appear in the file (e.g. the first artistsort is always paired to the first artist). This would be a pretty major change to how sort tags are interpreted AFAIK, but I think it makes sense.

In the case where there are more sort tags than base tags, I'd propose just ignoring the extra sort tags. That seems more sensible given the purpose of sort tags and the general interpretation that's being proposed.

@pfps

This comment has been minimized.

Contributor

pfps commented Jan 17, 2016

I don't think that I am proposing a change in how sort tags are associated with their base tags, actually. In places where multiple tag values are concatenated (e.g., in song lists) there will be no change. In places where multiple tag values result in multiple entries (e.g., in the paned browser) I don't think that sort tags are used at all so, yes, I am proposing a change here but this has already been acknowledged as a bug and the change is not in the way that sort tags work (because they aren't even used right now).

Are there other places in quodlibet where multiple tag values result in multiple entries? I am not aware of any. If there are, are sort tags used?

@lazka

This comment has been minimized.

Member

lazka commented Jan 17, 2016

If you mean how to handle sort values that are empty strings, then these probably should be treated just as for missing sort values, i.e., the display value is used as the sort value.

If you mean how to handle display values that have several different sort values, for example when showing a pane, then that's already a problem and should be handled the way it is now.

I mean the latter. This currently isn' t handled a all. I guess we should use the most common sort value one per row in this case.

@pfps

This comment has been minimized.

Contributor

pfps commented Jan 17, 2016

Using the most common value would be best, I think, but that's somewhat expensive.
I expect that the most common case where there are different sort values will be where some of the display values have a sort value and some don't. The solution here is to use the sort value. This may require a slight change to my proposal above.
Multiple sort values may be so uncommon that the simple solution of using the first one is sufficient.

@pfps

This comment has been minimized.

Contributor

pfps commented Jan 19, 2016

I did a preliminary implementation of sorting in the paned browser using sorting as in my long message above. The changes were smaller than I expected, probably because the only place that uses the _separate functions for tags is the paned browser, so changing their how they work didn't require any changes elsewhere.

I have a current version of my changes at https://github.com/pfps/quodlibet
I haven't put in a pull request for them yet as I may first try think up how to better handle multiple sort versions of a tag

@zsau

This comment has been minimized.

Contributor

zsau commented Jan 20, 2016

I'd say that if QL adopts this interpretation for list contexts then it also makes sense to do so in single-value (concatenation) contexts. I.e. if a song has two artists ["A", "B"] but only one artistsort ["C"] then we should use "C, B" as the concatenated sort value for the artist tag. The advantage would be consistent handling of sort tags in both contexts; only disadvantage AFAICT would be a potentially more disruptive change to QL's behavior.

Though it's not clear to me how many users would ultimately be affected by such a change. How common is it to have different numbers of base/sort tags, and what behavior would users actually expect or want in that case?

@pfps

This comment has been minimized.

Contributor

pfps commented Jan 21, 2016

Yes, having the same behaviour in concatenation contexts would be better. However, the change would probably not happen often and would often have only a minor or even no change on the sort order, because the two sort values would be C and C, B, which sort very close together.

@pfps

This comment has been minimized.

Contributor

pfps commented Jan 21, 2016

#1796 is a pull request for the necessary changes to make sort tags work in the paned browser. It does not include upgrades to sorting in other places.

@lazka

This comment has been minimized.

Member

lazka commented Feb 18, 2016

Is there anything left to do here?

@pfps

This comment has been minimized.

Contributor

pfps commented Feb 18, 2016

Sorting in the paned browser is done, I think.

Sorting in song lists should probably get some attention. I'm not sure that my changes there are complete fixes. I spent a bit of time trying to figure out some changes that don't affect so much of Quod Libet. I'll let you know of any significant progress.

I just took a look at the code in songlist.py. There appears to be two ways of determining whether a tag is a tag pattern there: the tag starts with a < and the tag contains a <. I think that the second is correct, which means that there is still a bug when sorting in song lists. Is my surmise correct?

@lazka

This comment has been minimized.

Member

lazka commented Feb 18, 2016

I just took a look at the code in songlist.py. There appears to be two ways of determining whether a tag is a tag pattern there: the tag starts with a < and the tag contains a <. I think that the second is correct, which means that there is still a bug when sorting in song lists. Is my surmise correct?

Sounds correct. Thanks for the summary.

@lazka lazka removed the needinfo label Feb 18, 2016

@pfps

This comment has been minimized.

Contributor

pfps commented Feb 18, 2016

OK. The bug would be exhibited in only very rare situations.
I'll put together a fix (and maybe even a test for it) in a separate pull request.

I think that this issue can be closed.

@lazka

This comment has been minimized.

Member

lazka commented Feb 18, 2016

Ok, thanks for working on this.

@lazka lazka closed this Feb 18, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment