Is your feature request related to a problem?
We need to ensure that the assistant provides relevant responses in SNEHA's program domains and avoids off-topic queries. This is crucial for maintaining the quality of interactions in real-world scenarios.
Describe the solution you'd like
- Run the topic relevance guardrail from
kaapi-guardrails on a randomly sampled subset of ~14,605 conversations of SNEHA.
- Start with 100 conversations to refine the prompt, then 500, and finally ~1.5k (~10% sampling).
- Document results since we don't yet have annotations or a known on-topic/off-topic ratio to stratify on.
Is your feature request related to a problem?
We need to ensure that the assistant provides relevant responses in SNEHA's program domains and avoids off-topic queries. This is crucial for maintaining the quality of interactions in real-world scenarios.
Describe the solution you'd like
kaapi-guardrailson a randomly sampled subset of ~14,605 conversations of SNEHA.