Skip to content

feat: add support for huggingface datasets#2497

Merged
typpo merged 8 commits intomainfrom
ian/20241222-154340
Dec 23, 2024
Merged

feat: add support for huggingface datasets#2497
typpo merged 8 commits intomainfrom
ian/20241222-154340

Conversation

@typpo
Copy link
Copy Markdown
Contributor

@typpo typpo commented Dec 22, 2024

Load test cases directly from HuggingFace datasets. To use a dataset, specify it in your config using the huggingface://datasets/ prefix. For example:

description: Testing with HuggingFace dataset

prompts:
  - 'Act as {{act}}. {{prompt}}'

providers:
  - id: openai:gpt-4o-mini

tests: huggingface://datasets/fka/awesome-chatgpt-prompts?split=train

This will load all rows from the dataset and use the act and prompt fields as variables in the prompt template.

@github-actions
Copy link
Copy Markdown
Contributor

Images automagically compressed by Calibre's image-actions

Compression reduced images by 26.5%, saving 330.62 KB.

Filename Before After Improvement Visual comparison
site/static/img/blog/beavertails/beavertails-eval.png 1.22 MB 917.70 KB -26.5% View diff

163 images did not require optimisation.

@typpo typpo merged commit f77a390 into main Dec 23, 2024
@typpo typpo deleted the ian/20241222-154340 branch December 23, 2024 05:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants