Skip to content

feat(cli): add initial promptfoo import command#1253

Merged
christso merged 3 commits into
mainfrom
feat/271-promptfoo-import
May 19, 2026
Merged

feat(cli): add initial promptfoo import command#1253
christso merged 3 commits into
mainfrom
feat/271-promptfoo-import

Conversation

@christso
Copy link
Copy Markdown
Collaborator

@christso christso commented May 19, 2026

Closes #271

Current promptfoo import scope:

  • adds agentv import promptfoo
  • converts common prompt/defaultTest/test shapes into AgentV YAML
  • supports inline and file-backed prompts plus YAML / JSON / JSONL / CSV test files
  • maps clean deterministic + rubric-based assertions
  • errors clearly on unsupported promptfoo features that still need a translation runtime
  • documents the supported v1 migration path in the import docs

Manual verification

Red (origin/main)

$ bun apps/cli/src/cli.ts import promptfoo /tmp/.../promptfooconfig.yaml --dry-run
error: found 1 error

  agentv import promptfoo /tmp/.../promptfooconfig.yaml --dry-run
                ^ Not a valid subcommand name

Green (this branch)

$ bun apps/cli/src/cli.ts import promptfoo ./promptfooconfig.yaml -o ./imported.eval.yaml
Imported promptfoo config → ./imported.eval.yaml

$ bun apps/cli/src/cli.ts validate ./imported.eval.yaml
Validation Summary
✓ ./imported.eval.yaml

Validation run on this branch

  • bun run build
  • bun run typecheck
  • bun run lint
  • bun run test

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented May 19, 2026

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: 1c0bbb3
Status: ✅  Deploy successful!
Preview URL: https://000098d7.agentv.pages.dev
Branch Preview URL: https://feat-271-promptfoo-import.agentv.pages.dev

View logs

christso and others added 2 commits May 19, 2026 20:55
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@christso christso marked this pull request as ready for review May 19, 2026 23:14
@christso christso merged commit 02923c2 into main May 19, 2026
4 checks passed
@christso christso deleted the feat/271-promptfoo-import branch May 19, 2026 23:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: promptfoo config import (promptfooconfig.yaml → EVAL.yaml)

1 participant