Skip to content

[Docs] Evals documentation missing some crucial context / information. #1391

@ssbushi

Description

@ssbushi

Is your report related to a problem? Please describe.
There are a few things that are glossed over the evaluation documentation (Evaluation and Writing an Evaluator Plugin)

Evaluator:

  • How to pass auth to the eval:flow command.
  • The format of the input.json file for eval:flow.

Writing an Evaluator Plugin:

  • Add more context to code snippets (eg: it is unclear if regexMatchScore and fillScores are provided in Genkit or if the user should implement)
  • Provide a simple explanation of the nature of EvaluatorFn.

Additional context
https://discord.com/channels/1255578482214305893/1309828408141807657

Some Genkit imports were missing / not updated to 0.9. This was addressed in this PR by @DenisVCode: #1388

Metadata

Metadata

Assignees

Labels

docsImprovements or additions to documentation

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions