LabelFewShot and Dspy.Suggest #1131
Description
Can I use LabeledFewShot with dspy suggestion? I have a bunch of labeled input/output pairs already. I want to compile use that program and do assertions on newly generated outputs. Am I thinking about this the right way? Since I have labeled example already I just need ensure the new outputs from the model are generated per a rule set.
Here is what I'm looking to do:
kbb_page_content = '''
Several things can caus ..
'''
carparts = '''
Codes that illuminate the MIL (Check Engine)
'''
rp_page_content = '''
OBD-II Code P0128 is
''''
KEYWORD ="p0128 code"
fewshot_examples = [
dspy.Example(keyword=KEYWORD, page_content=kbb_page_content, title="p0128 Code - Coolant Thermostat (Coolant Temperature Below Thermostat"),
dspy.Example(keyword=KEYWORD, page_content=carparts,title='P0128 Code: Coolant Temperature Below Thermostat Regulating Temperature')
]
class GenerateTitle(dspy.Signature):
"""Generate an SEO-friendly title based on a keyword and page content."""
keyword = dspy.InputField(desc="The main keyword to include in the title")
page_content = dspy.InputField(desc="The content of the page")
title = dspy.OutputField(desc="The generated SEO-friendly title")
# Define the DSPy module
class SEOTitleGenerator(dspy.Module):
def __init__(self):
super().__init__()
self.generate_title = dspy.ChainOfThought(GenerateTitle)
def forward(self, keyword, page_content):
title = self.generate_title(keyword=keyword, page_content=page_content).title
# # Assertions
dspy.Suggest(keyword_density(title, keyword) >= 2, "Keyword density should be at least 2%")
dspy.Suggest(title_length(title) <= 60, "Title length should be less than or equal to 60 characters")
dspy.Suggest(title_readability(title) >= 60, "Title readability should be at least 60")
dspy.Suggest(title_emotion(title)[0] >= 0, "Title should have a positive sentiment")
dspy.Suggest(title_relevance(title, page_content) >= 0.5, "Title relevance should be at least 0.5")
return dspy.Prediction(title=title)
seo_title_suggest = SEOTitleGenerator().activate_assertions()
trained = LabeledFewShot().compile(student=seo_title_suggest, trainset=fewshot_examples)
trained(keyword=KEYWORD,page_content=rp_page_content)