How do you set a name for a provider when it's coming from a separate file?

[vhpoet]

Currently, I can write:

providers:
  - openai:gpt-3.5-turbo
    id: openai-gpt-3.5-turbo-hightemp
    config:
      temperature: 0.9

This means the model is openai:gpt-3.5-turbo, and the name of the model that's also visible in the output HTML is openai-gpt-3.5-turbo-hightemp.

If the provider is coming from a file like this one:

providers:
  - file://providers/gpt-3.5-turbo-lowtemp.yaml

Then the provider file looks like this:

id: openai:gpt-3.5-turbo
config:
  temperature: 0.2

Here, the ID refers to the model, but I couldn't find a way to specify the name. So, what ends up happening is that I don't see the names of the models in my HTML output anymore.

[typpo]

Thanks for raising this, you're right that it's an issue when you include the provider in the way that you described. I'd like to add a label field or similar to the provider object, and get rid of the map that is used right now to label providers. That will make it easy for you to add a label.

[typpo]

#563 adds this capability, e.g. with the following setup:

promptfooconfig.yaml:

prompts:
  - 'Respond to the following instruction: {{message}}'

providers:
  - file://lowtemp.yaml
  - file://hightemp.yaml

tests:
  - vars:
      message: What's the capital of France?
  # ...

lowtemp.yaml:

id: openai:gpt-3.5-turbo-0613
label: openai-gpt-3.5-turbo-lowtemp
config:
  temperature: 0

hightemp.yaml:

id: openai:gpt-3.5-turbo-0613
label: openai-gpt-3.5-turbo-hightemp
config:
  temperature: 1

[typpo]

Released in 0.48.0!