Skip to content

Add system prompt to in-house judges#133

Merged
martinscooper merged 1 commit intomainfrom
system-prompt
Oct 27, 2025
Merged

Add system prompt to in-house judges#133
martinscooper merged 1 commit intomainfrom
system-prompt

Conversation

@martinscooper
Copy link
Copy Markdown
Contributor

@martinscooper martinscooper commented Oct 22, 2025

Changes:

  • Criteria's prediction_field was renamed to to_evaluate_field .
  • DirectInstance and PairwiseInstance models were removed and unified under the Instance model.
  • Instance now just holds a fields attribute. context, response and responses fields were removed.
  • The logic behind how the text to be evaluated and how the context is evaluated was re-designed. Now, the criteria defines the role of each of the instance fields.
  • Lanchain usage was heavily reduced and replaced by custom logic and the dependency will be removed soon.
  • EvalAssist in-house judges prompts were changes: the prompts are now using system prompts and the message format.
  • Lanchain's output fixer were replaced with custom logic.
  • Synthetic instance generation was improved.
  • More tests were added.

@martinscooper martinscooper force-pushed the system-prompt branch 2 times, most recently from 259a584 to ddc4ac1 Compare October 27, 2025 15:43
Signed-off-by: Martín Santillán Cooper <marsancoo@gmail.com>
@martinscooper martinscooper merged commit 072d03c into main Oct 27, 2025
8 checks passed
@martinscooper martinscooper deleted the system-prompt branch October 27, 2025 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant