Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use pydantic-yaml to save on token costs? #654

Closed
lingster opened this issue May 8, 2024 · 1 comment
Closed

Use pydantic-yaml to save on token costs? #654

lingster opened this issue May 8, 2024 · 1 comment

Comments

@lingster
Copy link

lingster commented May 8, 2024

In order to save on costs, could we implement a version of this that makes llm respond in yaml? This may also have a benefit of faster response times as less tokens would need to be generated and returned.

Perhaps this could be added as a flag?
The trick would be to ensure we can correctly validate the yaml is in a format that matches the pydantic model.

@jxnl
Copy link
Collaborator

jxnl commented May 8, 2024

this would work if we used mode=MD_YAML, happy to take a pr on this, and add it into the benchmarks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants