Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: LLMEvaluator uses OpenAI API with the the seed parameter #7720

Merged
merged 1 commit into from
May 21, 2024
Merged

feat: LLMEvaluator uses OpenAI API with the the seed parameter #7720

merged 1 commit into from
May 21, 2024

Conversation

davidsbatista
Copy link
Contributor

@davidsbatista davidsbatista commented May 21, 2024 •
edited

Related Issues

Proposed Changes:

  • when the LLMEvaluator uses OpenAI API make the call with the the seed parameter set, to force more determinism

How did you test it?

  • run several evaluations + LLMEvaluator tests

@github-actions github-actions bot added the type:documentation Improvements on the docs label May 21, 2024
@davidsbatista davidsbatista added the ignore-for-release-notes PRs with this flag won't be included in the release notes. label May 21, 2024
@coveralls
Copy link
Collaborator

coveralls commented May 21, 2024 •
edited

Pull Request Test Coverage Report for Build 9173969873

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.001%) to 90.553%
Totals Coverage Status
Change from base Build 9129529675: 0.001%
Covered Lines: 6585
Relevant Lines: 7272
💛 - Coveralls

@davidsbatista davidsbatista marked this pull request as ready for review May 21, 2024 11:33
@davidsbatista davidsbatista requested a review from a team as a code owner May 21, 2024 11:33
@davidsbatista davidsbatista requested review from shadeMe and removed request for a team May 21, 2024 11:33
@davidsbatista davidsbatista merged commit e6db150 into main May 21, 2024
35 of 36 checks passed
@davidsbatista davidsbatista deleted the add-seed-parameter-LLM-Evaluators branch May 21, 2024 13:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shadeMe shadeMe shadeMe approved these changes

Assignees
No one assigned
Labels
ignore-for-release-notes PRs with this flag won't be included in the release notes. type:documentation Improvements on the docs
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Make use of the seed parameter in the LLM-based evaluators using the OpenAI API
3 participants
@davidsbatista @coveralls @shadeMe