Allow plugins to add field types information for flexattrs

[pscn]

As discussed in #424 here's a proposal based on @sampsyo suggestion. It allows plugins to define field types.

In the plugin __init__ method you can now define the field types like so:

self.field_types['rating'] = float
self.field_types['play_count'] = int
self.field_types['skip_count'] = int

int and float will use the NumericQuery. Every other value will fallback to the SubstringQuery. A rudimentary conflict detection is also in place which will output something like this:

conflicting field type mapping detected. "liveness" declared by "echonest" as "<type 'float'>" and by "mpdstats" as "<type 'int'>"

A few examples how this looks like in action:

beet ls -f '$liveness: $artist - $title' liveness:0.791..0.792
0.791427063558378: Be Your Own Pet - Food Fight!
0.791255853844374: Beastie Boys - Heart Attack Man
0.791988194947472: Cliffy and Jerry - Looking for My Baby
[ ... ]
beet ls -f '$energy $danceability: $artist - $title' energy:0.95..0.99 danceability:0.8..
0.950648388205985 0.862588555530577: Buzzthrill - Everybody in the House
0.960362105796824 0.829378961009966: Los Calzones - Ska del papanata
0.968092780894187 0.826505789137516: Console - Gulls Galore
0.976002526222867 0.852790149199074: Cypress Hill - Psychodelic Vision
0.968634341909428 0.804499237628178: Depeche Mode - The Sun & the Rainfall
[ ... ]
beet ls -f '$play_count: $artist - $title' play_count:1..2
1: Curtis Lee - Pretty Little Angel Eyes
1: The Marvelettes - Beechwood 4-5789
[ ... ]

Would this be doable?

[sampsyo]

This looks awesome; thanks for putting this together. This kind of type map is exactly right; ideally, we can reuse the same infrastructure to do formatting as well (which we can tackle later, of course).

My only suggestion for the design here is to reduce the distinction between plugin-provided (flexattr) and built-in types. Maybe field_types can be constructed to contain both sets of types to make the type-to-query mapping simpler.

[sampsyo]

This looks awesome; thanks for putting this together. This kind of type map is exactly right; ideally, we can reuse the same infrastructure to do formatting as well (which we can tackle later, of course).

My only suggestion for the design here is to reduce the distinction between plugin-provided (flexattr) and built-in types. Maybe field_types can be constructed to contain both sets of types to make the type-to-query mapping simpler.

[pscn]

I just "hacked" the BooleanQuery to work with flexattrs and added support for quering template_fields to BooleanQuery and NumericQuery.

As an example I added the template_field played to mpdstats. It just checks if last_played is greater then 0 and returns True (or False if not set). I also added played as bool to field_types.

Now you can query like this:

beet ls -f '$played $last_played $artist - $title' played:True
True 1383226085 59 Times the Pain - Let Me In
True 1383221451 59 Times the Pain - Sense of Right and Wrong
True 1383312879 ART-SCHOOL - foolish
True 1383224504 Against All Authority - No Reason
[ ... ]

These are heavier changes then I initially intended, so I'm not sure if they have any unwanted side effects.

[pscn]

I just "hacked" the BooleanQuery to work with flexattrs and added support for quering template_fields to BooleanQuery and NumericQuery.

As an example I added the template_field played to mpdstats. It just checks if last_played is greater then 0 and returns True (or False if not set). I also added played as bool to field_types.

Now you can query like this:

beet ls -f '$played $last_played $artist - $title' played:True
True 1383226085 59 Times the Pain - Let Me In
True 1383221451 59 Times the Pain - Sense of Right and Wrong
True 1383312879 ART-SCHOOL - foolish
True 1383224504 Against All Authority - No Reason
[ ... ]

These are heavier changes then I initially intended, so I'm not sure if they have any unwanted side effects.

[pscn]

Another commit to complete mpdstats. Adding template_fields for all values returning defaults, if no flexattrs are set. Also adding a start_count template_field that just adds play_count and skip_count and makes it accessible through queries.

Examples for the default values:

beet ls -f '$start_count $rating $play_count $skip_count $last_played $artist - $title' start_count:0
0 0.5 0 0 0 +44 - Baby Come On
0 0.5 0 0 0 +44 - Make You Smile
[ ... ]

Example for start_count:

beet ls -f '$start_count $rating $play_count $skip_count $last_played $artist - $title' skip_count:1.. start_count:1..
3 0.6609375 2 1 1383228619 The Bouncing Souls - Old School
2 0.5390625 1 1 1383222249 The Lawrence Arms - Quincentuple Your Money
1 0.3125 0 1 0 Me First and the Gimme Gimmes - Isn't She Lovely

[pscn]

Another commit to complete mpdstats. Adding template_fields for all values returning defaults, if no flexattrs are set. Also adding a start_count template_field that just adds play_count and skip_count and makes it accessible through queries.

Examples for the default values:

beet ls -f '$start_count $rating $play_count $skip_count $last_played $artist - $title' start_count:0
0 0.5 0 0 0 +44 - Baby Come On
0 0.5 0 0 0 +44 - Make You Smile
[ ... ]

Example for start_count:

beet ls -f '$start_count $rating $play_count $skip_count $last_played $artist - $title' skip_count:1.. start_count:1..
3 0.6609375 2 1 1383228619 The Bouncing Souls - Old School
2 0.5390625 1 1 1383222249 The Lawrence Arms - Quincentuple Your Money
1 0.3125 0 1 0 Me First and the Gimme Gimmes - Isn't She Lovely

[pscn]

Finishing this up for today with adding album support.

Example:

beet ls -a -f '$play_count $skip_count $rating $albumartist - $album' play_count:1..
2 0 0.5375 59 Times the Pain - Blind Anger & Hate
1 0 0.5625 ART-SCHOOL - REQUIEM FOR INNOCENCE
1 1 0.501395089286 Various Artists - Mailorder for the Masses
[ ... ]

[pscn]

Finishing this up for today with adding album support.

Example:

beet ls -a -f '$play_count $skip_count $rating $albumartist - $album' play_count:1..
2 0 0.5375 59 Times the Pain - Blind Anger & Hate
1 0 0.5625 ART-SCHOOL - REQUIEM FOR INNOCENCE
1 1 0.501395089286 Various Artists - Mailorder for the Masses
[ ... ]

[sampsyo]

Looking very cool; thanks for all the revisions!

One thing we should take somewhat slow here: the way you've let queries access template fields here is very interesting—it makes them into something like "computed fields", which I think is a great direction. However, I think it's important that we tackle that in a general way. Specifically, it should be possible for any type of query to access any template field. This can work in the same way that we handle flexattrs (with "slow" queries that work transparently independent of the specific query kind). That way, substring and regex queries can get the same benefits.

We can either try to get to this ideal, orthogonal feature implementation in this PR or try splitting that off into a new branch of work—finish up the field-type stuff and then move onto computed fields. Got a preference?

[sampsyo]

Looking very cool; thanks for all the revisions!

One thing we should take somewhat slow here: the way you've let queries access template fields here is very interesting—it makes them into something like "computed fields", which I think is a great direction. However, I think it's important that we tackle that in a general way. Specifically, it should be possible for any type of query to access any template field. This can work in the same way that we handle flexattrs (with "slow" queries that work transparently independent of the specific query kind). That way, substring and regex queries can get the same benefits.

We can either try to get to this ideal, orthogonal feature implementation in this PR or try splitting that off into a new branch of work—finish up the field-type stuff and then move onto computed fields. Got a preference?

[pscn]

If I got some time tonight I'll try to make this work for all queries. Shouldn't be too much effort.

[pscn]

If I got some time tonight I'll try to make this work for all queries. Shouldn't be too much effort.

[sampsyo]

Okay, cool. My suggested tactic: add the functionality to LibModel itself (or to Album and Item if that's impossible). Then make obj[field] work for these fields. This could help simplify the template-filling code also.

[sampsyo]

Okay, cool. My suggested tactic: add the functionality to LibModel itself (or to Album and Item if that's impossible). Then make obj[field] work for these fields. This could help simplify the template-filling code also.

[pscn]

This is not forgotten. But I didn't find time to work on it. Hopefully over this week I get it done.

[pscn]

This is not forgotten. But I didn't find time to work on it. Hopefully over this week I get it done.

[sampsyo]

There's no rush, of course. 😃 Thanks for looking into it, but do take your time.

[sampsyo]

There's no rush, of course. 😃 Thanks for looking into it, but do take your time.

[geigerzaehler]

Superseded by #951. @pscn @sampsyo: Can we delete the branch

[geigerzaehler]

Superseded by #951. @pscn @sampsyo: Can we delete the branch