Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement EvalQueryEngineTool #11679

Merged
merged 6 commits into from
Mar 25, 2024

Commits on Mar 12, 2024

  1. Implement EvalQueryEngineTool

    Implement the EvalQueryEngineTool:
    * Inherits from QueryEngineTool
    * Uses a given evaluator to evaluate query engine responses
    * If the response fails evaluation:
        * The tool output is manipulated to deter the LLM
          from using it
        * The evaluator feedback is used as a reason for failure
    d-mariano committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    54e8b25 View commit details
    Browse the repository at this point in the history

Commits on Mar 18, 2024

  1. Configuration menu
    Copy the full SHA
    f99a927 View commit details
    Browse the repository at this point in the history
  2. Simplify eval_query_engine_tool imports

    * Simplify eval_query_engine_tool imports and avoid circular dependencies
    d-mariano committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    e2f0ea1 View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2024

  1. Fix EvalQueryEngineTool notebook

    * Fix EvalQueryEngineTool notebook by increasing similarity_top_k to 5
    * The above resulted in the lyft query engine to return a response
      that passed evaluation
    d-mariano committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    f1f2949 View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2024

  1. merge main

    logan-markewich committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    47da90b View commit details
    Browse the repository at this point in the history
  2. update docs

    logan-markewich committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    3cd2f37 View commit details
    Browse the repository at this point in the history