Is "label_and_rate" doing what it should do?

[QuiteChridi]

I just started out using fabric (great project btw) and one of my first ideas was to classify my (far too big) "watch later" playlist on youtube, using the "label_and_rate" pattern. After a few videos I felt like there was a lot of bias to AI specific content, which made me take a look at the pattern, where I found the following:

2. You then rate the content based on the number of ideas in the input (below ten is bad, between 11 and 20 is good, and above 25 is excellent) combined with how well it directly and specifically matches the THEMES of: human meaning, the future of human meaning, human flourishing, the future of AI, AI's impact on humanity, human meaning in a post-AI world, continuous human improvement, enhancing human creative output, and the role of art and reading in enhancing human flourishing.
3. Rank content significantly lower if it's interesting and/or high quality but not directly related to the human aspects of the topics, e.g., math or science that doesn't discuss human creativity or meaning. Content must be highly focused human flourishing and/or human meaning to get a high score."

This made me think about whether this side-effect of rating content about AI and philosophy higher is intended or some sort of side effect. You could of course make the argument that for personal consumption topics which are too technical are probably not that relevant and that AI is ubiquitous and therefore should be represented in content you consume on a daily basis.

I could also see this fitting especially Daniels (and to some extent his communities) needs because these topics are essentially what he is concerned with and what his work is about. I feel like it induces the risk of narrow-thinking and staying in some sort of confirmation-bubble (not because arguments against ubiquitous AI would be rated poorly but simply because content that has no AI focus would not be consumed if one is following the "label_and_rate" recommendations"

Tell me what you guys think about this. Is "label_and_rate" biased? Should it be biased? Or am I over thinking the whole topic and works pretty well in practice?

[pacocartones]

The themes are hard-coded in the pattern, and they weigh in more than one place. From reading data/patterns/label_and_rate/system.md on main (cef213cd9c; I haven't scored real content with it):

• Step 2 (line 43) combines the idea count with how well the content matches a fixed THEMES list: human meaning, human flourishing, the future of AI, AI's impact on humanity, and so on.
• Step 3 (line 45) and step 6 (line 59) both push content down if it's high quality but "not directly related to the human aspects", with math and science as the examples.
• The tiers (lines 51-55) and the 1-100 score (line 57) are defined against "the THEMES in STEP added missing import #2", and step 7 (line 61) scores content very low when it has neither interesting ideas nor any link to the THEMES.

So a strong technical video with plenty of ideas but no human-meaning angle would score low under the prompt as written. Whether that's intended is for the pattern's author to say.

If you want the same structure with your own priorities, you don't need to fork. Fabric has a custom patterns directory (fabric --setup → Tools → Custom Patterns), and per the README a custom pattern with the same name takes precedence over the built-in one and survives --updatepatterns. Copy label_and_rate/system.md there, replace the THEMES list in step 2, and drop or soften steps 3 and 6. Your playlist then gets ranked against your list instead.

[QuiteChridi]

Thanks for your answer. It's not that I don't know how to adapt it to my need. I just wanted to spark a discussion on whether a pattern with such a simple name should have such a big opinion.

You wouldn't expect a pattern called "check_academic_writing" to use some very specific standard of a single university, would you?

Essentially what I wanna discuss is as you put it "So a strong technical video with plenty of ideas but no human-meaning angle would score low under the prompt as written. Whether that's intended is for the pattern's author to say."

[pacocartones]

Short version: the same themes appear in 9 of the 265 patterns, the names don't mark which ones, and two descriptions say different things. I checked the latest release, v1.4.524, which is the same commit as main right now.

The set of interests (human meaning, human flourishing, AI's impact on humanity) is in label_and_rate and rate_content, as part of the scoring, and in the opening line ("interested in insights related to…") of extract_wisdom, extract_wisdom_nometa, extract_wisdom_with_attribution, extract_wisdom_agents, extract_wisdom_dm, extract_insights and extract_insights_dm. So it is a shared set of themes, not only one pattern.

The names don't separate it: there is a _dm suffix, which looks like a personal-variant convention, but the unsuffixed extract_wisdom uses the same themes.

Where the themes are stated differs. data/patterns/pattern_explanations.md says label_and_rate "rates it based on idea count and relevance to human meaning, AI, and other related themes". The one-line description in scripts/pattern_descriptions/pattern_descriptions.json is "Categorize/evaluate content by assigning labels and ratings.", and that is the file the web UI's pattern list loads (the web build copies it into static/data), so someone choosing from that list would not see the themes.

Making the themes a parameter isn't free either. Patterns take variables (-v=#name:value, as translate does with {{lang_code}}), but a missing variable is an error ("missing required variable", internal/plugins/template/template.go) and there is no default value, so a {{themes}} inside label_and_rate would make calls without -v fail. That makes the easiest fix a breaking change.

Two options that wouldn't break anyone, if the maintainers wanted to address it:

1. State the themes in the pattern_descriptions.json line for these patterns, where people choose.
2. Split by name: keep the current prompts under their names and add neutral siblings. (Moving the themed ones under the existing _dm convention would also separate them, but it renames patterns people already call, so it is a breaking change.)

Whether the generic names should carry these themes is still Daniel's call.