Create Quality Profiles and Quality Profile Groups with Recyclarr

[voltron4lyfe]

Is there an existing issue for this?

• I have searched the existing issues

Feature Description

To implement the TRaSH guidance, you need to manually configure Quality Profiles, group and order Qualities, and ensure cutoff scores, desired languages, etc... are all configured correctly. This setup is error prone and tedious to keep up-to-date when you have multiple Radarr/Sonnar instances and want to have "!" versions of profiles to keep the two instances in sync.

The proposal is to add "Quality Profile" and "Quality Group" definition to the yml config.

The suggested behavior:

1. The quality profiles are "declarative". Meaning the groups for the quality profile will be erased and re-created per the Recyclarr config. This is similar behavior toreset_unmatched_scores and delete_old_custom_formats. If a property is not defined in the quality_profile config (e.g. language), then the property will not be updated or, when creating, will be set to the Radarr/Sonarr default is. The quality profile itself must not be deleted as that would change the quality profile's id which would break lists and other functions.
2. If a quality profile doesn't exist then it will be created.
3. Any quality profiles that not defined in the recyclarr configs will not be deleted or updated.

Example:

radarr:
  quality_groups:
    - name: "Bluray|WEB-1080p"
      qualities:
        - "Bluray-1080p"
        - "WEBRip-1080p"
        - "WEBDL-1080p"
        - "Bluray-720p"
    - name: "WEB 720p"
      qualities: 
        - "WEBRip-720p"
        - "WEBDL-720p"
  
  quality_profiles:
    - name: "SQP-1"
      language: Original
      upgrades_allowed: True
      upgrade_until: "Bluray|WEB-1080P"
      minimum_cf_score: 10
      upgrade_until_cf: 10000
      qualities:
        - Bluray|WEB-1080p
        - WEB 720p

[yammes08]

Do you think quality_groups should be nested within quality_profiles? E.g.:

radarr:
  quality_profiles:
    - name: "SQP-1"
      language: Original
      upgrades_allowed: True
      upgrade_until: "Bluray|WEB-1080P"
      minimum_cf_score: 10
      upgrade_until_cf: 10000
      quality_groups:
        - name: "Bluray|WEB-1080p"
          qualities:
            - "Bluray-1080p"
            - "WEBRip-1080p"
            - "WEBDL-1080p"
            - "Bluray-720p"
        - name: "WEB 720p"
          qualities: 
            - "WEBRip-720p"
            - "WEBDL-720p"

[voltron4lyfe]

@rcdailey and I discussed that early on and we felt that the groups could be referred to by multiple quality profiles so it made sense to split them up as concepts.

Another option might be to move the quality groups to outside of the "service" definition (e.g. to a top level) that way the same quality groups could be used by different services (e.g. radarr-4k & sonarr-4k could use the same). Not sure though, what do you think?

[rcdailey]

A few thoughts:

We might also want to provide a way to enable ungrouped qualities as well. I'd expect the "groups" list to represent the ordering of qualities you see in the UI while editing a quality profile. Anything not mentioned in the list won't be checked. Items in the list will be sorted to the very top in the same order they appear under groups.

Obviously this means groups probably isn't the right name:

radarr:
  quality_groups:
    - name: "Bluray|WEB-1080p"
      qualities:
        - "Bluray-1080p"
        - "WEBRip-1080p"
        - "WEBDL-1080p"
        - "Bluray-720p"
    - name: "WEB 720p"
      qualities: 
        - "WEBRip-720p"
        - "WEBDL-720p"
  
  quality_profiles:
    - name: "SQP-1"
      language: Original
      upgrades_allowed: True
      upgrade_until: "Bluray|WEB-1080P"
      minimum_cf_score: 10
      upgrade_until_cf: 10000
      qualities:
        - Bluray|WEB-1080p
        - WEB 720p
        - HDTV-720p

The above is almost identical to your original example, except groups is qualities instead. It accepts either fixed quality names (as they are known by the service) or custom group names, and uses this lookup order:

• Names are first checked for matches in the quality_groups list.
• If no match, then check against the hard-coded list of qualities provided by the service.
• If no match, error.

The example above uses HDTV-720p directly, which is not a group. And I'd expect that to work. The only thing I don't like about this is that it's a bit confusing. How do you know what's a group or not from the array itself? I'm not sure if that's a big deal, though.

@rcdailey and I discussed that early on and we felt that the groups could be referred to by multiple quality profiles so it made sense to split them up as concepts.

I'd add to this that it's also an attempt to reduce nesting. Indentation can get out of control with some of these configurations. Breaking things out not only makes them reusable but also keeps things easier to read (structurally).

Another option might be to move the quality groups to outside of the "service" definition (e.g. to a top level) that way the same quality groups could be used by different services (e.g. radarr-4k & sonarr-4k could use the same). Not sure though, what do you think?

I think I'd solve this problem with YAML includes. The topic of having includes has come up a few times in the past. I've always avoided doing the work because I strongly feel that the UI should be doing this. Basically, in the UI I'd expect groups to be centralized (or: not specific to any one service type) and reference them anywhere. The resulting YAML spit out by the GUI application would duplicate the data in the necessary spots.

[yammes08]

The example above uses HDTV-720p directly, which is not a group. And I'd expect that to work. The only thing I don't like about this is that it's a bit confusing. How do you know what's a group or not from the array itself? I'm not sure if that's a big deal, though.

What would happen in the event that you call your quality group "HDTV-720p" as well? That's possible in the UI - would you have to include a check to see if the config has defined a group with that name, and use that if it is, instead of the standalone quality?

I'd add to this that it's also an attempt to reduce nesting. Indentation can get out of control with some of these configurations. Breaking things out not only makes them reusable but also keeps things easier to read (structurally).

Fair enough. To me it seems more logical nesting it as part of a profile but I understand the reasons for not wanting to do it that way, but...

I think I'd solve this problem with YAML includes. The topic of having includes has come up a few times in the past. I've always avoided doing the work because I strongly feel that the UI should be doing this. Basically, in the UI I'd expect groups to be centralized (or: not specific to any one service type) and reference them anywhere. The resulting YAML spit out by the GUI application would duplicate the data in the necessary spots.

...but saying that, I like this approach. This I think is actually more consistent with the Custom Formats themselves; Custom Formats are completely external in their composition, so when you add one to your config file you're saying 'pull this external thing into my config file'. Having an external file or files containing quality groups would conceptually be very similar.

[voltron4lyfe]

The example above uses HDTV-720p directly, which is not a group. And I'd expect that to work. The only thing I don't like about this is that it's a bit confusing. How do you know what's a group or not from the array itself?

Agree that it should be: qualities and not groups in the quality_profile. That was my mistake and I've updated the original post. The logic could be "for each quality, see if there is a group with that name and, if not, then assume it's a standalone quality".

If you still wanted the quality to appear but not be enabled (because you want to control how manual search results are displayed), then you could include it and there could be an optional enabled property when the quality is referenced in the quality_profile.

Proposed example (and sorry in advance @rcdailey that it adds more indentation):

  quality_profiles:
    - name: "SQP-1"
      language: Original
      upgrades_allowed: True
      upgrade_until: "Bluray|WEB-1080P"
      minimum_cf_score: 10
      upgrade_until_cf: 10000
      qualities:
        - Bluray|WEB-1080p
        - WEB 720p
        - HDTV-720p
          enabled: false 

[rcdailey]

What would happen in the event that you call your quality group "HDTV-720p" as well? That's possible in the UI - would you have to include a check to see if the config has defined a group with that name, and use that if it is, instead of the standalone quality?

It would use the group without question. Direct access to the quality (instead of the group) in this case would be impossible. However, I'd expect that to not matter since presumably you have that quality within the group you're referencing. If not, then there's a naming issue and I would choose different names.

...but saying that, I like this approach. This I think is actually more consistent with the Custom Formats themselves; Custom Formats are completely external in their composition, so when you add one to your config file you're saying 'pull this external thing into my config file'. Having an external file or files containing quality groups would conceptually be very similar.

I'm open to reconsidering include directives in the YAML but it's not at the top of my priority list sadly.

Agree that it should be: qualities and not groups in the quality_profile. That was my mistake and I've updated the original post. The logic could be "for each quality, see if there is a group with that name and, if not, then assume it's a standalone quality".

I wouldn't necessarily call this your "mistake". I didn't go back and read our old Discord conversation, but this might just be a new corner case I didn't think of. In either case, I appreciate you giving this so much thought!

If you still wanted the quality to appear but not be enabled (because you want to control how manual search results are displayed), then you could include it and there could be an optional enabled property when the quality is referenced in the quality_profile.

Proposed example (and sorry in advance @rcdailey that it adds more indentation):

  quality_profiles:
    - name: "SQP-1"
      language: Original
      upgrades_allowed: True
      upgrade_until: "Bluray|WEB-1080P"
      minimum_cf_score: 10
      upgrade_until_cf: 10000
      qualities:
        - Bluray|WEB-1080p
        - WEB 720p
        - HDTV-720p
          enabled: false 

I don't think that's valid YAML If you're using a mapping, all elements must be key value pairs; so you'd have to do something like this instead:

         - name: HDTV-720p
           enabled: false 

But honestly, I don't like that: It's too verbose and cluttered. That's why I proposed the order of precedence in the lookup semantics. I'm leaning towards making these kinds of corners a user-naming issue and have them fix it. It surely isn't that difficult to select more unique names for groups.

[voltron4lyfe]

Doh! You are correct, I thought there was a way to have a default key but I don't think so.

I was trying to capture another case. What I learned recently is that even if a quality is not enabled for a profile, it still affects the weighting of the manual search results. So people may want to include disabled qualities and specify their order. Let me think on it.

[rcdailey]

affects the weighting of the manual search results

Can you explain this in greater detail?

[yammes08]

affects the weighting of the manual search results

Can you explain this in greater detail?

Manual search results are shown in order of the listed qualities/quality groups regardless of whether they're selected or not, and then by custom format score within each quality/quality group.

It's also worth bearing in mind that it can affect what files are downloaded. Let's say you already have a 1080p file in your movie folder. You set up a new profile, and only have Bluray-720p selected, and ordered below all the 1080p qualities in the list. In this scenario no file will ever be downloaded, because the existing file beats the highest selected quality, even though the 1080p quality is itself not selected.

[rcdailey]

Manual search results are shown in order of the listed qualities/quality groups regardless of whether they're selected or not, and then by custom format score within each quality/quality group.

So the way I was thinking this would be implemented in Recyclarr is as follows. For any group or quality in the YAML:

• Enable it (check the box)
• Sort it to the top of the list

For example, let's say I have four qualities: a, x, y, z. And by default, Radarr sorts them like this:

• a
• x
• y
• z

Now, in my YAML I only specify x and z in this order:

• z
• x

Here is what would happen:

1. z and x get checked (enabled)
2. a and y get unchecked (disabled)
3. Any enabled items get moved to the top. All unchecked items appear at the bottom
4. Checked items are sorted in the order shown in the YAML, while still remaining above unchecked items.

The final list would look like this:

• z
• x
• a
• y

Given this, I don't think your first concern is an issue. Because if manual search results appear based on the order of the qualities and not their checked/unchecked status, the ordering is still managed by Recyclarr in such a way that the ones you care about (i.e. the checked qualities/groups) will appear on top above the unchecked.

It's also worth bearing in mind that it can affect what files are downloaded. Let's say you already have a 1080p file in your movie folder. You set up a new profile, and only have Bluray-720p selected, and ordered below all the 1080p qualities in the list. In this scenario no file will ever be downloaded, because the existing file beats the highest selected quality, even though the 1080p quality is itself not selected.

You said you sorted 720p below 1080p, so I think this behavior is expected. Aren't you telling Radarr that you prefer 1080p over 720p? Why would it replace the 1080p one with a 720p copy in this case?

One other thing you're leaving off is the cutoff quality which can also impact this behavior (well, it would prevent the search in the first place IIRC).

At the end of the day I don't see how the logical ordering I described above would cause an issue here. User is still in full control of whether or not 1080p qualities are above or below 720p.

[yammes08]

Agreed on all points - I didn't think any of these things were issues, was just trying to describe @voltron4lyfe 's comment in some more detail.

The second comment about 1080p/720p and selection/non-selection isn't really anything to do with recyclarr I suppose, it was just tangentally related. I'd seen discussion elsewhere this week that a user could hypothetically already have a movie file, set up a profile expecting an 'upgrade' based on CF scores, and then not see anything happening even though an 'unwanted' quality being unchecked. You're right that cut-off quality factors into this too.

Anyways, in short, I agree these aren't really issues, and even if they were then your ordering proposal would make them non-issues in any event.

[rcdailey]

I 100% appreciate bringing up all the points you did. I'm not really trying to "shoot down" issues; rather I like to talk through them and make sure an idea is robust enough to not have those problems. It was good to be able to think through things. Thanks!

[yammes08]

Absolutely, and it hasn't come across as such!