Adding yaml prompt support to other models

[therealpaulgg]

Hey,

I was trying to use the new yaml prompt file feature but realized that it's currently only supported with OpenAI.

I see this is the code that does it:

 const trimmedPrompt = prompt.trim();
    if (trimmedPrompt.startsWith('- role:')) {
      try {
        // Try YAML
        messages = yaml.load(prompt) as { role: string; content: string }[];
      } catch (err) {
        throw new Error(
          `OpenAI Chat Completion prompt is not a valid YAML string: ${err}\n\n${prompt}`,
        );
      }
    } else {
      try {
        // Try JSON
        messages = JSON.parse(prompt) as { role: string; content: string }[];
      } catch (err) {
        if (
          process.env.PROMPTFOO_REQUIRE_JSON_PROMPTS ||
          trimmedPrompt.startsWith('{') ||
          trimmedPrompt.startsWith('[')
        ) {
          throw new Error(
            `OpenAI Chat Completion prompt is not a valid JSON string: ${err}\n\n${prompt}`,
          );
        }

        // Fall back to wrapping the prompt in a user message
        messages = [{ role: 'user', content: prompt }];
      }
    }

Wondering if this code should be moved out to a preprocessing layer before the provider is called, that way it would be called regardless of the provider. Thoughts?

[typpo]

Thanks Paul - this went out in version 0.17.6!