Skip to content

Conversation

@habuma
Copy link
Member

@habuma habuma commented Jun 27, 2024

This is similar to #948, but applied to EvaluatorRequest. Aside from being about to use the evaluator at runtime, this makes it possible to write a test like this:

@Test
public void testAsk() {
  String question = "Why is the sky blue?";
  Answer answer = askService.ask(new Question(question));

  RelevancyEvaluator evaluator = new RelevancyEvaluator(chatClientBuilder);
  EvaluationResponse response = evaluator.evaluate(
      new EvaluationRequest(
          question,
          List.of(),
          "Scattering of light causes the sky to be blue."));

  Assertions.assertThat(response.getScore()).isEqualTo(1.0f);
}

@markpollack
Copy link
Member

updated docs
merged in b5cc9db

@markpollack markpollack closed this Jul 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants